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CHAPTER  I 


X  NTRODUCT I ON 


Purpose 


The  purpose  of  this  study  is  to  investigate  the  effects 
of  different  voice  types  (male,  female,  and  machine)  on  the 
accuracy  and  speed  of  response  to  a  voice  warning  system. 
The  results  of  the  study  will  be  used  in  the  design, 
development,  and  selection  of  auditory  advisory  annunciator 
systems  for  military  aircraft.  Therefore,  the  moderating 
effects  of  different  background  levels  of  noise,  auditory 
level  of  the  warnings,  and  warning  message  formats  will  be 
introduced.  Parameters  that  are  not  specifically  listed  as 
variables  will  maintain  constant  values  typical  for 
military  aircraft. 


Backaround 


In  the  last  few  years  the  number  and  variety  of  warning 
signals  in  modern  high-perf ormance  aircraft  have  grown 
steadily.  For  example,  a  Federal  Aviation  Administration 
(FAA)  research  project  found  that  "in  going  from  the  B-707 
to  the  B-747,  the  number  of  alerting  signals  increased  from 
188  to  455,  or  142  percent"  (Berson  et  al . ,  1981,  p.  2). 
The  F-14A  has  47  separate  warning  and  caution  signals 
(Butler  et  al . ,  1981).  In  the  military  cockpit,  this 
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problem  is  compounded  with  the  addition  of  combat  alerting 
signals  in  addition  to  normal  alerts.  Especially  under 
high  stress  combat  situations,  the  audio-visual  load  on  the 
pilot  may  reach  saturation  level,  potentially  diminishing 
per-formance  (Thorburn,  1981;  Kemerling,  et  al .  ,  1969). 

The  primary  -functions  o-f  an  alert  system  include: 
attracting  crew  attention,  identifying  the  urgency  of  the 
alert,  and  providing  information  as  to  the  adequacy  of  the 
corrective  action  (Berson  et  al . ,  1981).  Ideally,  the  time 
required  to  detect  and  evaluate  alert  conditions  should  be 
minimized,  as  well  as  the  time  required  to  begin  corrective 
action,  all  while  maintaining  safety  of  flight  and  threat 
avoidance  (Boucek  et  al . ,  1981). 

There  are  four  priorities  of  alerts  that  warning 
systems  bring  to  the  attention  of  the  pilot/crew: 

1.  Warning:  Emergency  operational  or 

aircraft  system  conditions  that  require 
immediate  corrective  or  compensatory  crew 
action; 

2.  Caution:  Abnormal  operational  or  aircraft 
system  conditions  that  require  i mmedi ate 

crew  awareness  and  require  prompt 
corrective  or  compensatory  crew  action; 

3.  Advisory:  Operational  or  aircraft  system 

conditions  that  require  crew  awareness  and 

may  require  crew  action;  and 

4.  Information:  Operational  or  aircraft 

system  conditions  that  require  cockpit 
indications,  but  not  necessarily  as  part  of  the 
integrated  warning  system  (Boucek  et  al . ,  1981, 
p .  3)  . 

Alerting  systems  that  identify  these  four  priorities 
can  be  categorized  into  three  basic  types:  visual,  aural, 
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and  tactile. 


Each 


of  these  categories  can  have  a  “master 


warning"  signal,  as  well  as  a  separate  signal  to  identify 
the  particular  type  of  alert  (Boucek,  Veitengruber  8<  Smith, 
1977) . 

As  currently  implemented,  visual  systems  use  different 
colors,  sizes,  and  intensities  of  lights  to  symbolize  the 
priority  of  the  alert  (red  -  warning;  yellow  -  caution; 
blue,  green,  or  white  -  advisory).  Also,  colored  lights, 
bands,  and  flags  are  used  to  further  specify  the  nature  of 
the  problem.  However,  there  are  no  standard  visual  alert 
systems,  so  that  similar  alerts  can  specify  different 
problems  on  different  aircraft  (Vei  tengruber ,  Boucek 
Smith,  1977). 

A  lack  of  standardizati on  is  readily  apparent  in  aural 
alerts  as  well.  Aural  alerts  can  take  the  form  of  bells, 
horns,  chimes,  tones,  clicks,  warblers,  and  voice,  ail  at 
varying  intensities  and  frequencies.  Each  alert  can 
identify  a  particular  problem;  but,  as  with  visual  alerts, 
each  alert  can  signify  different  problems  on  different 
aircraft  ( Vei tengruber ,  Boucek  &  Smith,  1977). 

Tactile  systems  have  been  restricted  to  inducing 
vibration  into  the  aircraft  yoke  as  an  attention  getting 
device  (Boucek,  Veitengruber  8<  Smith,  1977).  Due  to  the 
skin's  inability  to  "compete  with  the  eyes  or  ears  in  the 
ability  to  make  fine  discriminations  or  transmit  complex 
information  rapidly"  the  use  of  tactile  systems  in  aircraft 
is  limited  in  the  amount  of  information  they  can  provide 


(Heard,  1970,  p.  31). 

The  prol if eration  ot  aierts,  especially  auditory 

alerts,  is  such  that  it  would  be  easy  for  a  crewmember  to 
miss  or  con-fuse  the  signal  of  an  emergency  condition.  To 
prevent  such  a  situation  it  has  been  recommended  to  reduce 
the  number  of  aural  alerts,  or  devise  a  scheme  which 
provides  more  information  for  each  alert  (Pollack  2 <  Tecce, 
1938;  Vei tengruber ,  Boucek  &  Smith,  1977).  One  method 

would  be  to  augment  the  auditory  signal  with  a  visual  text 
display  of  the  specific  nature  of  the  problem,  such  as 
could  be  provided  on  a  cathode  ray  tube  (CRT).  Another  way 
would  be  to  replace  the  current  multitude  of  bells, 

buzcers,  beeps,  etc.  with  a  voice  warning  system  (VWS) . 
The  voice  warning  could  specify  the  nature  of  the  alert, 
thereby  providing  more  information  and  reducing 
mi sunderstandi ngs.  Williams  and  Simpson  (1976)  reported  a 
British  Airways  paper  that  argued  strongly  against  the  use 
of  nonverbal  aural  alerts,  since  they  are  limited  in 
information  content  and  can  be  startling  or  distracting. 
Voice  alerts  are  recommended  for  high  priority, 

quick-action  alerts.  Williams  and  Simpson  also  recommended 
a  visual  signal,  with  appropriate  color  code  to.  di sti ngui sh 
between  warnings,  cautions,  and  advisories,  as  a  backup  to 
all  auditory  signals. 

In  their  pamphlet  on  recommended  practices  for  flight 
deck  signals,  the  Society  of  Automotive  Engineers  stated 
that  a  unique,  attention-getting  sound  (such  as  a  chime. 
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etc.)  together  with  voice  alert  messages, 


provide  a  warning  superior  to  that  o-f  discrete 
aural  alerts.  There-fore,  it  is  strongly 
recommended  that  discrete  aural  alerts  not  be 
used  in  an  integrated  -flight  deck  alerting 
system.  I-f,  however,  -familiarity  and/or  usage 
should  make  desirable  the  use  o-f  certain  discrete 
aural  alerts,  they  shall  be  limited  to  a  maximum 
o-f  tour,  and  shall  meet  international  standards 
to  be  agreed  to  (1980,  p.  2). 

A  poignant  example  ot  the  contusion  and 

mi sunderstandi ngs  that  can  be  caused  by  the  prol i terat i on 
ot  nonstandardized  warnings  is  the  recent  incident 
involving  a  C-141  crew  (A  Big  Misunderstanding,  1982). 
While  taking  ott  trom  a  CONUS  (Continental  United  States) 
base,  still  on  takeott  roll,  the  stall  warning  horn  and  the 
master  caution  light  came  on.  None  ot  the  crewmembers 
recognized  the  horn.  The  copilot  initially  thought  it  was 
an  improper  contiguraton  warning  horn,  until  he  realized 
that  that  horn  was  only  on  Boeing  727's;  he  then  decided  it 
was  sate  to  take  ott,  still  not  knowing  what  the  horn  meant 
on  the  C-141. 

An  unknown  crewmember  said  "reject".  The  aircratt  was 
stopped  only  300  test  short  ot  the  end  ot  the  overrun. 

An  investigation  showed  that  the  horn  did,  indeed, 
sound  remarkably  similar  to  the  improper  contiguration  horn 
on  the  727  (the  pilot  and  jump-seat  pilot  were  both  current 
in  the  727;  the  copilot  had  been  current  in  the  727  about 
ten  years  earlier).  In  addition,  ot  the  105  crewmembers 
later  asked  to  identity  the  seven  audible  warning  signals 
in  the  C-141,  only  7  crewmembers  correctly  identitied  all 


■;  j 


» 


the  signals.  A  survey  also  determined  that  most  crews  (of 

» 

the  11  surveyed)  ignored  the  horn  when  they  heard  it  5 
knots  before  "go"  speed.  "Go"  speed  is  the  speed  at  which 
the  pilot  is  committed  to  take  off,  since  it  would  be 
unsafe  to  stop. 

There  are  two  distinct  advantages  of  a  voice  warning 
system.  One  is  that,  especially  for  "heads-up"  flying, 
responses  to  voice  messages  are  faster  than  responses  to 
tone  signals  (Kemmerling  et  al . ,  196?)  or  visual  signals 
Ooucek,  Veitengruber  &  Smith,  1977;  Davis,  Rundle  & 
Stockton,  1981;  Cooper,  1977).  "Heads-up"  flying  refers  to 
the  current  practice  of  flying  in  which  the  pilot's 
attention  is  focused  out  of  the  cockpit.  To  affect  this 

technique,  as  much  information  as  possible  (airspeed, 

altitude,  direction,  etc.)  is  presented  in  heads-up 
displays  (HUD's)  so  that  the  pilot  need  not  focus  his 
attention  down  into  the  cockpit  (Butler  et  al . ,  1981). 

Davis,  Rundle  and  Stockton  (1981)  noted  that  several 
mid-air  collisions  have  been  caused  by  pilots  trying  to 
perform  system  checks  while  flying  in  close  formation. 

The  second  advantage  is  that  specific  voice  alert 
information  gives  the  pilot  the  option  of  evaluating  the 
alert  and  responding  immediately,  or  of  delaying  his 
response,  secure  "in  the  knowledge  that  it  would  be  safe  to 
do  so"  (Davis,  Rundle  ?<  Stockton,  1981,  p.  vii).  Thus,  if 
his  current  task  is  of  extraordinary  priority  (threat 
avoidance),  he  need  not  neglect  that  task  in  order  to 


identify  and  evaluate  the  alert,  probably  with  unpleasant 
results. 


Hi  story 

The  Air  Force  began  experimenting  with  voice  warning 
systems  (VWS)  in-  1961,  when  a  successful  audio  tape  system 
was  introduced  into  the  entire  fleet  of  B-58  Hustlers 
(Davis,  Rundle  Sj  Stockton,  1931;  Thorburn ,  1971). 

Subsequent  questioning  by  the  Directorate  of  Aerospace 

Safety  showed  that  91  of  97  pilots  felt  that  the  VWS 
contributed  to  flight  safety,  and  all  but  two  wanted  the 
VWS  in  the  FB-111  if  they  were  assigned  to  that  aircraft 
(cited  in  Kemmerling  et  al . ,  1969). 

In  1963,  the  Tactical  Air  Command  performed  a  series  of 
tests  using  a  VWS  in  the  F-100F  aircraft.  The  results 
showed  a  significant  improvement  in  pilot  reaction  time, 
especially  under  heavy  loading  or  stress  (Thorburn,  1971). 
In  fact,  the  improvement  factor  was  42-to-l  during  the 
highest  workload  phase  of  the  test  (Davis,  Rundle  St 
Stockton,  1981).  Cooper  (1977)  stated  that  the  Air  Force 

found  a  six  to  nine  second  improvement  in  warning 

recognition  through  the  use  of  a  VWS.  Favorable  results 
were  also  found  in  tests  at  the  Naval  Air  Test  Center  in 
1963  (Kemmerling  et  al . ,  1969). 

The  results  of  the  1963  VWS  tests  prompted  the  Air 
Force  Inspector  General  for  Air  Safety  (AFIAS)  to  state 
that  his  office  "..firmly  supports  the  installation  of 
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voice  warning  systems  in  all  hi  gh-perf ormance  aircraft 
wherein  there  is  no  -flight  engineer  position"  (cited  in 
Thorburn ,  1971,  p.  3).  The  AFIAS  letter  cited  several 
incidents  in  which  the  VWS  had  prevented  a  serious  accident 
(Kemmerling  et  al . ,  1969). 

The  Army  also  became  heavily  involved  in  testing  the 
use  of  VWS.  In  1968,  a  study  at  the  Aberdeen  Proving 


Ground  identified 

the 

characteri st i cs 

of  a  VWS 

for 

installation  into 

six 

Army  aircraft. 

including 

five 

helicopters  (Brown,  Bertone  &  Dbermayer ,  1968). 

Despite  the  clear  advantages  of  voice  warning  systems, 
implementation  in  aircraft  has  been  slow.  Lea  (1981) 
noted: 

Listening  to  machines  may  be  difficult  in  the 
presence  of  auditory  noises  or  interfering 
conversations.  The  spoken  utterance  is  not 
readily  sustained  in  time,  such  as  are  lights, 
displays,  or  manual  knob  positions.  Speech 
generated  by  machines  may  not  be  attended  to,  or 
understood  by,  the  human  listener,  and  if  not 
heard  when  initially  spoken,  it  will  not  be 
continuously  available  for  later  scrutiny.  If 
the  machine  does  not  speak  intelligibly  and  with 
good  voice  quality,  these  problems  may  be 
particularly  accented  <p.  3). 

Other  objections  cited  by  Pollock  (1958)  include  such 
factors  as  cost,  weight,  and  reliability.  However,  changes 
in  technology  have  overcome  these  objections.  The 
necessary  hardware  and  circuitry  now  are  quite  compact, 
with  significant  improvements  in  reliability.  For  example, 
SCI  Systems,  Inc. ,  manufactures  a  voice  warning  system  for 
the  F-14  aircraft  that  occupyies  476  cubic  inches  and 
weighs  11  pounds;  Rockwell  International  manufactures  a 


system  called  FPA-SO,  Flight  Path  Advisory  System,  which 
occupies  93.5  cubic  inches  and  weighs  only  3  pounds; 
McDonnel  1  -Dougl as  manu-f actures  a  voice  warning  generator 
•for  aircraft  that  occupies  90  cubic  inches  and  weighs  4 
pounds  (Butler  et  al . ,  1981). 

An  additional  reason  for  the  delay  in  implementation 
may  have  been  air  tactics  in  the  Vietnam  conflict.  It  was 
common  for  missions  to  have  a  large  number  of  aircraft 
attacking  a  single  target.  Such  missions  achieved 
coordination  by  everyone  using  the  same  radio  frequency. 
The  addition  of  a  VWS  was  one  more  voice  that  pilots  did 
not  want  to  hear  (Davis,  Bundle  8c  Stockton,  1981). 

Types  of  Speech  Synthesis 

Improvements  in  speech  synthesis  technology  have 
alleviated  early  concerns  about  the  practicality  and 
reliability  of  voice  warning  systems.  Speech  generation 
requires  that  the  phonetic  characteri sti cs  of  words  or 
parts  of  words  be  stored  in  a  digital  form  so  that  they  can 
be  retrieved  and  reproduced  by  a  speech  synthesis  device. 
Werkowitz  (1980)  and  Davis,  Bundle  and  Stockton  (1981)  note 
that  three  current  methods  of  speech  coding  and  synthesis 
are  waveform  coding,  linear — predictive  coding  (LF'C)  ,  and 
phoneme  synthesis. 

Waveform  coding  is  a  technique  in  which  speech  is 
sampled  and  the  digitized  samples  are  stored  in  Bead  Only 
Memory  (ROM) .  The  data  are  compressed  so  that  an 


9 


intelligible  signal  is  produced  when  the  coded  speech  data 
are  processed  by  a  wave-form  synthesizer  which  may  contain  a 
di gital /analog  converter.  Waveform  coding  generates  a  very 
high  quality  voice  signal  but  requires  a  relatively  large 

amount  of  computer  memory. 

0 

Linear  predictive  coding  analyzes  speech  to  determine 
pitch,  amplitude,  and  frequency  characteristics  as  a 
function  of  time.  These  characteri sti cs  are  then  stored  in 
ROM.  Reproduction  of  the  voice  is  performed  by  using  a 
multi-stag'?  digital  filter,  a  pulse  generator,  and  a  random 
noise  generator.  The  quality  of  the  voice  that  is 
generated  is  not  as  good  as  in  waveform  coding,  but  because 
this  method  stores  less  information  about  the  voice,  less 
computer  memory  is  required. 

Phoneme  synthesis  forms  words  or  phrases  by  connecting 
basic  speech  sounds  (phonemes).  Phoneme  synthesis  requires 
very  little  ROM.  The  process  involves  encoding  the  basic 
speech  sounds  only,  and  producing  intelligible  speech  by 
sequentially  reproducing  different  combinatins  of  the 
stored  data.  The  English  Language  can  be  synthesized  by 
using  between  45  and  60  phonemes  or  phoneme  variations 
(Butler  et  al . ,  1981).  One  manufacturer  stated  that  150 
bits  per  second  can  generate  continuous  speech.  However, 
the  speech  has  a  very  unnatural ,  mechanical  sound 
(Werkowitz,  1980;  Davis,  Rundle  &  Stockton,  1981). 


Problem  Statement 


Increasing  aircraft  complexity  and  pilot/crew  workloads 
have  increased  the  importance  of  timely  and  appropriate 
reactions  to  warning  signals  or  messages-  There  are 
favorable  experimental  results  that  have  shown  that  voice 
warning  systems  decrease  reaction  times  significantly 
without  severe  impact  on  the  primary  task  of  flying 
(Vei tengruber ,  Boucek  §<  Smith,  1977).  The  problem  is  that 
there  are  insufficient  data  to  indicate  whether  a  male, 
female,  or  machine  voice  should  be  used. 

■3  us't  i  f  i  c  at  i  on 

In  the  Aircraft  Alerting  Systems  Criteria  Study, 

Volume _ I,  the  need  for  data  collection  on  certain 

factors  affecting  "Verbal  Auditory  Caution  and  Warning 
Signals"  is  identified.  Voice  type  was  rated  "Highest 
priority  data"  (Vei tengruber ,  Boucek  &  Smith,  1977,  p.  80). 

Though  several  surveys  among  pilots  indicate  a 
preference  for  the  female  voice  (Berson  et  al . ,  1981),  no 
study  could  be  found  which  measured  accuracy  and  speed  of 
response  as  a  function  of  the  type  of  voice.  There  is  a 
need  for  an  empirical  study  to  determine  whether  the 
expressed  preference  is  justified,  or  whether  another  voice 
type  might  be  better  in  terms  of  alert  response. 


Research  Question 


This  experiment  will  investigate  whether  there  is  a 
significant  difference  in  the  accuracy  and  speed  of 
response  to  different  voice  types  under  various  background 
noise  levels,  si gnal -to-noi se  ratios,  and  warning  formats 
in  a  simulated  military  aircraft  cockpit  noise  environment. 


Scope  and  Limitations 


The  research  will  be  limited  to  investigations  of  the 
following  differences: 

1.  Voice  types:  male,  female,  and  machine; 

2.  Warning  format:  tone  precursor,  voice  precursor, 
and  repeated  warning; 

3.  Background  noise:  105  db  and  115  db;  and 

4.  Signal— to-noi se  ratio:  0,  5,  and  10  db  above 

headphone  conversation  noise. 

Speed  of  response  will  consider  only  accurate 
responses. 


Assumpti ons 


A  voice  warning  in  a  cockpit  environment  will  normally 
be  given  to  a  crewmember  who  is  proficient  at  his  primary 
task  (flying  the  aircraft)  and  capable  of  performing  a 
secondary  task  (responding  to  the  alert).  Subjects  for 
this  experiment  were  specifically  trained  in  the  primary 
and  .  secondary  tasks  to  assure  a  common  level  of 
proficiency.  This  was  done  to  ensure  sufficient  task 
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knowledge  to  prevent  any  training  effects  -from  coun-founding 
the  data.  Also,  task  proficiency  was  considered  more 
critical  than  job  title;  subjects  were  therefore  not 
necessarily  crewmembers,  but  were  personnel  who  have  passed 
the  rigorous  hearing  and  training  requirements  necessary 
for  this  experiment.  Personnel  at  the  Biological  Acoustics 
Branch  of  the  Aerospace  Medical  Research  Laboratory  at 
Wri ght-Patterson  Air  Force  Base,  Ohio  stated  that  "previous 
data  indicates  that  there  is  a  high  correlation  between  the 
reactions  of  the  pilot  population  and  the  general 
population"  (Anderson  8<  McKinley,  1993). 

It  was  further  assumed  that  most  warnings  require  a 
manual  reponse  by  the  crewmember,  such  as  flipping  a  lever, 
pushing  a  button,  turning  a  knob,  etc.  The  secondary  task 
in  this  experiment  therefore  required  a  manual  response. 

Previ ew 

Chapter  II  contains  a  literature  review  focusing  on  the 
response  to  different  voice  types  in  aircraft  alerting 
systems.  Methodology  is  discussed  in  Chapter  III, 
including  background  information  on  the  moderating 
variables.  T.he  equipment  used  and  experimental  design  are 
presented,  followed  by  an  explanation  of  the  methods  of 
data  reduction.  The  results  and  analysis  of  the  data  are 
listed  in  Chapter  IV.  Conclusions  are  presented  in  Chapter 
V,  and  a  general  discussion  and  recommendations  are  offered 
in  Chapter  VI. 
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CHAPTER  II 


1 _ I  TERATURE  FREE V  I  EEUJ 

There  is  a  marked  scarcity  o-f  literature  comparing 
voice  types,  especially  regarding  voice  warning  systems. 
Only  two  studies  were  -found  that  address  di -f Terences  in 
reactions  to  male  and  -female  voices. 

In  1948,  Black  and  Graybiel  investigated  the  e-f-fect  ot 
heard  stimuli  on  spoken  responses.  Research  showed  that 
the  subjects  reacted  to  di-f-ferent  voice  types  with  reponses 
o-f  di-f-ferent  intensities.  The  subjects  responded  in  a 
louder  voice  to  the  female  voice  as  compared  with  their 
response  to  a  male  voice.  However,  the  results  may  be 
misleading  because  all  the  subjects  were  male.  In 
addition,  Black  and  Graybiel  noted  that  the  observed 
differences  might  be  due  to  "non-identical  intensity  levels 
in  the  stimulus  materials"  <1948,  d.  1).  In  the  other 
study  comparing  voice  types,  Kerce  (1979,  cited  in  Berson 
et  al . ,  1981)  tested  the  relative  intelligibility  of 
different  voice  types.  In  her  study,  Kerce  measured 
intelligibility  by  having  subjects  record  verbatim 
reproducti ons  of  verbal  messages.  Results  indicated  that 
the  female  voice  was  more  intelligible  than  the  male  voice 
and  that  both  were  preferable  to  a  synthetic  "machine" 
voice.  Kerce  recommended  that  although  the  female  voice 


14 


will  generally  be  mare  intelligible  than  the  male  voice, 
voice  character! sti c  selection  should  be  based  on  a 
spectral  analysis  of  the  voice  model  and  ambient  noise 
environment  in  order  to  achieve  the  best  possible  results 
(Berson  et  al . ,  19S1). 

In  the  analysis  or  VWS  -for  implementation  in  Army 
aircraft,  Brown,  Bertone  and  Obermayer  (1968)  recommended 
that  a  -female  voice  be  used.  The  recommendation  was  based 
on  vehicle  noise  characteri st i cs  (frequencies  of  engine 
noise)  as  well  as  pilot  reports  which  claim  that  the  female 
voice  would  be  more  distinctive  and  therefore  more  easily 
heard  over  other  communications. 

That  the  female  voice  is  less  common  in  voice 
communications  and  thus  more  distinctive  is  a  prevalent 
argument  for  using  the  female  voice  (Berson  et  al . ,  1981). 
Davis,  Bundle  and  Stockton  (1981)  recommended  a  female 
voice  both  because  of  its  higher  register  and  its 
distinctiveness  in  communications.  They  stated  that 
despite  more  and  more  female  voices  heard  over  the  radio, 
the  majority  of  voices  are  still  male. 

Butler  et  al .  (1981)  acknowledged  the  supposed 
"differentness"  of  the  female  voice  as  the  reason  that  that 
voice  was  used  in  early  voice  warning  systems.  However, 
they  suggested  that  the  current  trend  toward  using  women  as 
controllers,  both  military  and  civilian,  effectively 
counters  that  argument.  Butler  et  al .  also  suggested  that 
there  may  be  some  pilots  who  do  not  want  to  hear  a  female 
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voice  while  in  action. 

A  voice  that  could  be  more  distinctive  than  the  -female 
voice  is  a  synthesized  neutral  or  machine  voice.  Such  a 
voice  would  be  more  unlike  normal  communications  than  a 
human  voice  and  thus  even  more  distinctive.  The  voice 
could  be  easily  generated  by  any  o-f  the  voice  synthesisers 
described  in  Chapter  I.  Simpson  and  Williams  (1930)  stated 
that  synthesized  speech  could  be  used  both  to  attract 
attention  and  to  per-form  the  in-formation  transmission 
-function. 

Factors  affecting  the  intelligibility  o-f  a  voice 
include  its  intensity  and  -frequency.  Nemeyer  (1981)  noted 
that  the  so-ftest  noise  the  ear  can  hear  is  caused  by 
pressure  changes  on  the  order  o-f  0.00002  Newtons  per  square 
meter.  The  -frequency  range  to  which  the  ear  is  sensitive 
ranges  from  30  hertz  to  18  kilohertz  in  young  people,  and 
from  about  100  hertz  to  10  kilohertz  in  older  people 
(Radio,  1974).  Muller  (1975)  stated  that  the  base 
frequency  of  female  voices  is  from  210  to  240  hertz  and 
that  the  base  frequency  of  male  voices  is  from  130  to  140 
hertz.  Brown,  Bertone  and  Obermayer  (1968)  noted  the  same 
difference  in  voice  frequencies  in  their  analysis  of  VWS 
for  the  Army.  They  further  stated  that  the  distinctiveness 
of  the  female  voice  may  be  related  to  its  higher  base 
frequency. 

Perceived  loudness  of  sound  is  determined  by  a 
combination  of  sound  frequency  and  intensity.  That  is,  for 
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two  tones  of  equal  intensity.  the  one  with  the  higher 
•frequency  will  be  perceived  as  louder  (Berson  et  al  .  ,  1981; 
Fletcher.  1953).  Besides  contributing  to  perceived 
loudness,  intensity  greatly  a-f-fects  intelligibility  (Black 
Zi  Graybiel  ,  1948).  However,  intensity  is  not  meaningful  in 

itself;  rather,  it  is  the  intensity  relative  to  the 
environment  noise  level  that  affects  perceived  loudness  and 
intelligibility. 

Davis,  Bundle  and  Stockton  (1981)  suggested  that  the 

intensity  of  the  alert  in  a  audible  warning  system  be 

preset  and  not  subject  to  pilot  control .  They  recommended 
a  level  of  105  db  as  high  enough  to  attract  attention  but 
not  so  loud  as  to  be  startling.  Other  researchers  have 
suggested  varying  the  alert  in  a  range  of  5  to  15  db  above 

ambient  noise  (Berson  et  al .  ,  1981;  Brown,  Bertone  S< 

Obermaver,  1968;  Boucek,  Veitenoruber  ?<.  Smith,  1977). 
However,  it  should  be  noted  that  the  consensus  among  pilots 
in  a  1977  survey  was  that  most  aural  alerts  in  commercial 
aircraft  were  too  loud  (Cooper,  1977). 

Studies  about  implementing  a  VWS  in  particular  aircraft 
(Brown,  Bertone  &  Obermayer,  1968;  Butler  et  al . ,  1981; 
Davis,  Bundle  8<  Stockton,  1981)  recommended  using  a  female 
voice  due  .to  its  perceived  distinctiveness  as  well  as  the 
relative  infrequency  of  female  voices  in  radio 
communications.  However,  distinctiveness  and 

intelligibility  of  the  voice  are  assets  in  the  cockpit  only 
to  the  extent  that  the  pi  1 ot /crewmember  will  be  able  to 
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respond  correctly  and  quickly  to  a  warning  signal .  The 
authors  of  this  thesis  were  unable  to  find  an  empirical 
study  of  response  to  different  voice  types  in  a  voice 
warning  system.  Thus,  despite  the  widely  accepted 

perception  that  the  female  voice  is  more  distinctive  in  a 
cockpit  environment,  there  is  a  need  to  determine,  by 
empirical  tests,  the  voice  type  to  which  people  respond 


most  accurately  and  quickly. 


CHAPTER  II J 


METHODOLOGY 

The  chapter  on  Methodology  is  divided  into  four 
sections.  Section  one  will  discuss  the  variables  to  be 
examined.  Section  two  will  explain  the  equipment  used  to 
perform  the  experiment  and  record  the  data.  Section  three 
will  explain  the  experimental  design.  Section  four  will 
review  the  data  reduction  and  analysis  techniques  used  to 
evaluate  the  results. 

Variables  to  be  Examined 

Dependent  variables  are  accuracy  and  speed  of  response. 
Independent  variables  are  male,  female,  and  machine  voice 
types.  Moderating  variables  are  background  noise,  masking 
by  background  conversation,  and  precursors  to  the  warning. 
Accuracy  of  Response 

The  required  response  to  each  warning  is  to  push  the 
correct  button,  as  specified  in  each  particular  warning. 
Accuracy  is  measured  as  a  ratio  of  the  number  of  correct 
responses  to  the  total  number  of  required  responses. 

Reaction  vs.  Response 

Reaction  time  is  the  time  it  takes  an  observer  to 
detect  a  signal  and  react  to  it  when  that  is  the  only  task 
that  he/she  is  required  to  do.  Time  measurements  which 
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reaction  time  are  ■  "not  contaminated  by  other 


variables,  such  as  workload  or  distracting  movements,  and 
are  therefore  the  optimum  response  unit"  (Boucek, 

Veitengruber  Smith,  1977,  p.  2). 

Response  time,  when  dealing  with  experiments  conducted 
in  a  real  or  simulated  cockpit  environment,  is  a  measure  o-f 

the  time  to  respond  to  a  signal  when  that  is  not  the  only 

thing  an  observer  is  doing.  Actually,  the  response  is 

o-ften  a  secondary  task  accomplished  simultaneously  with  the 
primary  task  (i.e.  ,  -flying  the  airplane).  Reaction  time 
can  give  "an  indication  as  to  the  direction  o-f  the  results 
•for  response  time,  but  it  is  not  necessarily  a  direct 
measurement"  (Soucek,  Veitengruber  8 <  Smith,  1977,  p.  2). 

To  make  the  results  o-f  this  experiment  more  relevant  to 
the  cockpit  environment  in  military  aircra-ft,  subjects  were 
given  simultaneous  tasks.  Tracking  a  blinking  light  was 
their  primary  task  and  responding  to  the  voice  warning  was 
their  secondary  task.  Response  time  is  de-fined  as  the  time 
•from  the  beginning  of  the  warning  message  to  the  time  it 
takes  to  complete  the  appropriate  response. 

Voice  Types 

Male  and.  female  voice  warnings  were  recorded  in  the 
anechoic  chamber  of  the  Biological  Acoustics  Laboratory  at 
Wri ght-F'atterson  Air  Force  Base,  Ohio.  The  recorded  voices 
were  subsequently  digitised  for  reproducti on .  The  machine 
voice  was  produced  by  removing  the  variation  in  the 
frequency  of  the  recorded  male  speaker. 
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Backcround  Noise 


Voice  intelligibility  varies  with  different  levels  of 
background  noise.  Kerce  (1979,  as  noted  in  Berson  et  al .  . 
1981)  used  a  background  noise  level  of  76  db  in  her 
investigation  o-f  voice  intelligibility.  In  an  experiment 
investigating  the  e-f-fects  o-f  precursors,  Simpson  and 
Williams  (1930)  used  a  cockpit  noise  level  which  measured 
75  db  average.  Many  commercial  aircra-ft  have  an  ambient 
noise  level  in  the  cockpit  around  76  db.  The  noise  levels 
in  military  aircra-ft  are  normally  higher.  For  example, 
in-flight  noise  levels  in  the  F-16  fighter  range  from  92  to 
112  db  (Hille,  1979).  Under  the  presumption  that  warning 
and  response  times  are  most  critical  when  the 

pi lot /crewmember  is  under  high  task-loading  and  high  noise 
conditions,  background  engine  noise  levels  of  105  db  and 
115  db  were  used  to  allow  the  results  of  this  experiment  to 
be  more  applicable  to  the  military  cockpit  environment. 
Masking 

Masking,  in  voice  communications,  occurs  when  a  person 
cannot  distinguish  between  a  particular  signal  (the  voice) 
and  background  noise.  Masking  can  be  complete  or  partial. 
In  partial  masking,  enough  of  the  signal  may  get  through  so 
that  it  is  still  intelligible. 

Berson  et  al .  (1981)  noted  that  with  a  background 
noise  level  of  abou~  30  db,  it  requires  a  smaller 
incremental  increase  in  amplitude  to  make  lower  frequencies 
discernible  than  to  make  higher  frequencies  discernible 
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over  the  background  noise,  in  the  base  -frequency  range  of 
the  human  voice.  The  male  voice  has  a  base  frequency 
around  130  hertz,  while  the  average  base  frequency  of  the 
female  voice  is  about  210  hertz  (Mul ler , 1975) .  Thus, 
although  several  sources  (Berson  et  al . ,  1981;  Boucek, 

Veitengruber  &  Smith,  1977;  Brown.  Bertone  &  Obermayer, 
1963;  Thorburn,  1971)  cite  preference  for  the  female  voice 
due  to  its  distinctiveness,  the  male  voice  may  be 
discernible  at  a  lower  si gnal -to-noi se  ratio  than  the 
female  voice. 

To  investigate  the  effects  of  masking,  the  voice 
warning  was  fed  into  the  subjects'  earphones  at  three 
signal-to-noise  (S/N)  levels:  0,  5,  and  10  decibels  above 
the  headphone  conversation  level. 

Precursor 

When  a  voice  alert  is  not  preceded  by  an  attention 
getting  sound,  it  is  possible  for  the  pi  lot /crewmember  to 
miss  the  first  few  syllables  of  the  voice  message.  A  1979 
Douglas  Aircraft  Company  survey  (Berson  et  al . ,  1981)  and  a 
study  by  Boucek  et  al .  (li-jl)  both  found  that  pilots 
overwhelmingly  prefer  an  aural  precursor  to  the  voice 
warning.  The  difference  between  those  preferring  a  tone 
precursor  to  those  preferring  voice  (e.g.,  "Warning"  or 
"Caution")  was  not  significant  (Boucek  et  al.,  1981). 

The  voice  warnings  in  the  baseline  Voice  Synthesis 
System  (VSS)  for  the  Navy's  F-14  Tomcat  precedes  voice 
alerts  by  the  word  "Warning"  or  "Caution"  (Butler  et  al . , 
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1981).  The  Voice  Message  System  on  the  Air  Force's  F-15 
Eagle  uses  the  same  -format  (Davis,  Rundle  &  Stockton, 
1981).  Brown,  Bertone  and  Obermayer  (1988)  recommended 
that  the  voice  alert  be  preceeded  by  an  aural  tone  for  the 
Army.  Kemmerling  et  al .  (1969),  however,  warned  that  the 

aural  precursor  might  actually  be  a  nuisance  or 

distraction.  Despite  this,  the  1974  edition  of 
MIL-STD-1472B ,  Human  Eng ineerng  Design  Criteria  for 

Military  Systems  f  Equipment ,  and  Facilities ,  specifically 
required  the  use  of  a  nonspeech  alerting  signal  prior  to  a 
voice  warning  (Simpson  &  Williams,  1980). 

Until  1980,  there  mss  no  experimental  evidence  to 
support  those  recommendations.  Simpson  and  Williams  (1980) 
performed  an  experiment  to  test  whether  a  tone  or  voice 
precursor  elicited  the  shorter  response  time.  They  found 
that  the  effect  of  a  tone  before  a  voice  warning  was  to 
actually  lengthen  the  response  time  (i.e.  subjects  reacted 
more  slowly).  When  the  voice  warning  was  lengthened  with 
another  word,  however,  the  response  time  did  not  increase. 

Studies  by  Simpson,  Hart,  and  Williams  between  1975  and 
1980  showed  that  commercial  airline  pilots  reacted  faster 
to  messages  in  full  sentence  format  than  to  messages  using 
key  words  or  short  phrases  (Davis,-  Rundle  &  Stockton, 
1981),  indicating  that  context  increases  the 

intelligibility  and/or  decreases  response  time.  However, 
when  the  question  was  asked  of  military  pilots  whether  they 
would  prefer  short,  military-style  messages  as  opposed  to 
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full  sentences,  the  response  was  unanimously  in  favor  of 
the  short,  military-style  messages  (Davis,  Rundle  & 
Stockton,  1981) 

ft  1963  Navy  test  of  a  voice  system  installed  in  the 
VA-3B  aircraft  indicated  that  response  times  to  warnings 
delivered  once  were  slower  than  to  those  warnings  using  a 
"double  call-out"  (warning  repeated).  The  difference  in 
rescnse  time  was  not  significant.  The  pilots  indicated 
that  they  preferred  the  double  call-out  (Lilleboe,  1963, 

p.  11). 

To  investigate  the  confounding  effect  of  precursors  on 
voice  warning  response  times,  three  delivery  styles  were 
used:  repeated  message  (double  call-out),  a  tone  precursor, 
and  an  aural  ("Warning")  precursor.  The  repeated  message 
was  not  preceded  by  any  other  alert,  since  the  message 
acted  as  its  own  precursor.  The  tone  precursor  had  a 
frequency  of  500  hertz  and  sounded. for  0.3  seconds;  it  was 
delivered  at  the  same  amplitude  as  the  voice  warning  (0,  5, 
and  10  db  above  the  ambient  conversation  over  the 
headsets) . 


Equipment  Used 


The  experiment  was  conducted  using  the  Voice 
Communication  Research  and  Evaluation  System  (V0CRES) , 
located  in  the  Biodynamics  and  Bioengineering  Division  of 
the  Aerospace  Medical  Research  Laboratory  at 


Wri ght-Patterson  AFB,  Ohio. 


The  V0CRES  was  developed  "to 


provide  the  capability  -for  comprehensive  research ,  test  and 
evaluation  activities  in  the  voice  communications 
e-F f ecti veness  arena"  (McKinley,  1980,  p.  2).  Ten  consoles 
are  located  in  a  large  reverberat i on  chamber;  the  chamber 
has  a  programmable  sound  source  capable  of  emulating 
aircraft  engine  noise  as  heard  in  the  cockpit.  Each 
console  is  equipped  with  a  standard  AIC-25  aircraft 
intercommuni cati on  system.  Voice  communication  is  affected 
through  standard  headsets  in  HGU-26/P  flight  helmets. 

Each  console  has  two  panels:  a  display-response  unit 
and  a  keyboard  for  communication  performance  task  response 
(see  figure  1).  Both  panels  are  connected  to  an  on-line 
computer  data  collection  and  analysis  system.  The  Central 
Processing  Unit  (CPU)  for  this  system  is  the  Hewlett 
Packard  (HP)  9845T  computer.  The  display /response  unit  was 
used  for  the  primary  tracking  task,  while  the  keyboard  was 
used  to  measure  the  secondary  warning  response  task.  For 
a  more  detailed  description  of  the  VOCRES  facilities,  see 
Appendix  A. 

Digitised  voices  were  generated  by  the  Texas 
Instruments  (TI)  5220  Speech  Synthesis  chip,  using  the 
linear  predictive  coding  (LF’C)  method.  Voice  warnings  and 
background  voice  communications  were  heard  through  the 
headsets  in  the  helmets.  Engine  noise  was  generated  via 
the  programmable  sound  source,  through  the  loudspeakers 


lining  the  room. 


m 
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Figure  1.  Console  in  VOCRES  racihty 


The  computer  -facilities  recorded  the  response  time  -for 


each  warning,  and  kept  a  record  o-f  the  sequence  o-f  the 


warnings. 


Time 


was  measured  usina  a  ZOO  CTC 


microprocessing  chip,  which  measures  to  an  accuracy  o-f  3 


microseconds. 


Delegation  of  responsibilities  for  calibration  and 


maintenance  of  the  equipment,  together  with  details  of  the 


experimental  procedures  for  VOCRES  personnel,  are  contained 


in  Appendix  B1  (Test  Plan). 


26 


Experimental  Design 


Primary  Task 

The  primary  tracking  task  was  performed  on  the  upper 
panel.  An  asterisk  was  displayed  on  the  screen,  and  the 
subject  had  to  "track"  the  asterisk  by  touching  the  button 
closest  to  the  symbol.  There  were  eight  buttons,  four  on 
each  side.  The  subjects  had  to  hit  the  appropriate  button 
each  time  the  asterisk  appeared,  or  it  was  recorded  as  an 
error  in  the  primary  task.  The  asterisk  was  displayed 
every  0.6  second  throughout  the  test,  which  lasted  about  35 
minutes  per  run.  To  ensure  that  subjects  performed  as  well 
as  possible  throuahout  each  session,  they  were  advised  that 
they  would  not  be  used  in  further  tests  if  their  accuracy 
fell  below  a  certain  level.  Since  the  pay  bonus  was  only 
effective  if  they  completed  all  the  sessions,  there  was  a 
built-in  incentive  to  do  well.  Trial  runs  conducted  prior 
to  the  actual  experiment  established  an  acceptable  level  of 
accuracy  for  this  task. 

To  increase  the  primary  task  loading  and  simulate  a 
busy  cockpit  environment,  background  tapes  of  aircrew 
conversations  and  radio  transmi ssi ons  were  piayed  through 
the  subjects'  headsets  throughout  each  run. 

Secondary  Task 

The  secondary  task,  which  was  to  respond  to  the  voice 
warning,  was  accomplished  on  the  lower  panel  of  the 
console.  The  lower  panel  is  made  up  of  two  keypads, 
containing  sixteen  buttons  each.  The  pads  are  set  side  by 
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side,  thus  producing  an  arrangement  of  -four  rows  and  eight 
columns.  Each  row  is  a  different  color  (red,  grey,  white, 
blue).  The  columns  are  numbered  "1"  through  ,l4,‘  on  each 
keypad.  The  warning  directed  the  subject  specifically 
which  button  to  push,  identifying  it  by  left-or — right, 
color  and  number. 

Experimental  Procedure 

The  experiment  was  of  a  balanced  design.  For  each 
session,  one  voice  type,  precursor  format,  engine 
background  noise,  and  signal -to-noi  se  ratio  were  used. 
Each  combination  was  accommodated  once;  that  means  there 
were  3  (voice  types)  *  3  (precursor  formats)  *  2  (engine 
background  noises)  *  3  (S/N  ratios)  =  54  trials.  The 

possible  combinations  are  depicted  in  Figure  2. 


Pre¬ 
cursor:  . REPEATED . TOME . VOICE . 

Backgrd 

Noise:  - 105 . 115 . 105 . 115 . 105 . 115 - 

S/N 

Ratio:  0  5  10  0  5  10  0  5  10  0  5  10  0  5  10  0  5  10 


MALE  *  »  i  ttt  ttt  t  »  *  t  *  i  t  t  t 

FEMALE  *  *  *  *  i  *  *  *  *  tit  tit  ttt 

MACHINE  *  *  *  *  *  *  ttt  ttt  ttt 


Figure  2.  Treatment  Combinations 


In  order  to  minimize  the  effect,  of  any  possible 
learning  curve,  the  combination  of  parameters  and  the 
original  order  of  the  runs  were  determined  using  random 


28 


i 


methods  o-f  selection.  The  random  order  thus  determined  is 


reflected  in  the  run  parameter  code.  The  order  o-f  the  runs 
was  then  adjusted  only  when  necessary  to  remain  within  USAF 
standards  -for  noise  exposure,  as  established  by  AFR  161-35, 
Hazardous  Noise  Exposure,  and  AFR  169-3,  Use  of  Human 
Subjects  in  RDT&E.  The  -final  sequence  is  indicated  by 
the  run  sequence  number  (see  Experimental  Design  Sequence, 
Appendix  B2)  . 

Ten  subjects  were  each  seated  at  a  console  in  the 
VOCRES  -facility,  wearing  the  flight  helmet.  Throughout  the 
session,  they  performed  the  primary  tracking  task,  which 
required  a  continuous  level  of  participation.  Warnings 
were  interjected  at  random  times  through  the  headsets  in 
the  helmet.  For  the  response  to  be  counted  as  accurate, 
the  subjects  had  to  push  the  correct  button  within  three 
seconds  after  the  end  of  the  warning.  The  primary  task  did 
not  stop  when  warnings  were  interjected,  but  continued  to 
require  attention.  Detailed  instructions  to  the  subjects 
are  contained  in  Appendix  B3. 

Subjects 

To  ensure  that  the  results  of  experiments  at  the  VOCRES 
laboratory  were  as  free  as  possible  from  the  contaminating 
effects  of  having  someone  learn  the  task  while  undergoing 
the  experiment,  subjects  used  in  VOCRES  tests  were  screened 
by  personnel  at  the  lab.  Each  subject  was  tested  for  any 
signs  of  hearing  loss.  Before  being  allowed  to  participate 
in  the  experiment,  subjects  underwent  about  twenty  hours  of 
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training,  to  ensure  proficiency  at  the  task.  Personnel  at 
the  lab  have  -found  that  in  speed  of  response  tests, 
subjects  with  inadequate  training  do  not  contribute  useable 
data  due  to  the  learning  effect  in  this  type  of  experiment 
(Anderson  &  McKinley,  1983).  Personnel  at  the  lab  maintain 
a  current  list  of  available  subjects  with  no  appreciable 
hearing  loss  who  are  familiar  with  VOCRES  procedures. 

Subjects  who  participated  in  this  experiment  were  paid 
on  an  hourly  basis.  The  pay  scale  had  a  built  in 
incentive,  in  that  the  hourly  rate  was  increased  if  the 
subjects  were  present  for  all  sessions  in  the  experiment 
and  provided  useable  data.  This  ensured  consistency  by 
using  the  same  subjects  throughout  all  54  runs. 

Of  the  ten  subjects  who  participated  in  this 
experiment,  four  were  male  and  six  were  female.  All  ten 
have  been  previously  employed  as  subjects  either  full  or 
part  time  at  the  VOCRES  facility.  The  ages  range  from  20  to 
40  years  old. 

Other  Considerations 

To  reduce  spatial  disorientation,  the  subjects  were 
allowed  to  choose  their  seats  during  the  training  sessions. 
Once  they  had  chosen  a  seat,  they  maintained  that  seat 
throughout  all  54  runs. 

To  reduce  any  possible  bias  due  to  the  physical 
location  of  the  response  buttons  on  the  lower  panel,  all  32 
possible  warnings  were  called  out  during  each  session.  The 
order  of  the  callings  was  set  by  a  random  number  generator, 
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and  was 

changed  for  each  new 

sessi on . 

By  requiring 

that 

all  32 

warnings  be  called  during  each 

run,  the  effect 

s  of 

certain 

word  combinations  (e.g. 

"right 

wni te. . . " ,  " . . . 

blue 

two"  , 

"right  red...",  etc.) 

were 

balanced  for 

the 

experiment . 

To  ensure  that  the  subject  did  not  sit  and  wait  -for  a 
rhythmic  sequence  of  warnings,  the  time  between  warnings 
was  also  set  by  a  random  number  generator,  with  the 
following  two  constraints:  no  closer  together  than  10 
seconds,  no  further  apart  than  5  minutes.  The  sequence  was 
initially  generated  about  a  normal  dstribution  with  a  mean 
of  one  minute,  establishing  approx i mate! y  a  half-hour  core 
session  time. 

"End  effects"  are  aberrations  of  responses  early  in  the 
session  (just  getting  warmed  up)  and  at  the  end  of  sessions 
(know  it  will  end  soon,  so  mind  is  not  on  experiment).  To 
reduce  end  effects,  an  additional  three-minute  segment  was 
added  at  the  beginning  of  the  session,  and  a  two  minute 
segment  was  added  at  the  end;  responses  during  these  end 
segments  were  not  included  in  the  analysis.  Subjects  were 
not  aware  of  the  addition  of  these  end-segments  nor  that 
the  results  of  the  two  end-segments  were  not  recorded. 

To  ensure  that  all  32  buttons  were  nit  in  the  core 
session  time,  warnings  were  generated  using  the  same  random 
number  algorithm  for  the  two  end  segments  as  was  used  for 
the  core  segment.  Thus  some  buttons  were  repeated  during 
the  session — once  during  the  core  session,  and  possibly  one 
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or  more  times  during  the  end  segments.  This  also  ensured 
that  no  subject  could,  like  a  good  poker  player,  "count 
cards"  and  anticipate  which  buttons  would  most  likely  be 
called  near  the  end  o-f  the  session. 

To  generate  a  baseline  -for  accuracy  o-f  the  primary 
task,  one  more  constraint  was  imposed:  a-fter  the  sequence 
and  time  intervals  -for  the  secondary  task  were  generated, 
the  longest  time  between  warnings  was  set  to  equal  -five 
minutes.  This  did  not  a-f-fect  any  other  time  in  the 
session.  The  -five  minute  time  was  used  each  session  to 
assess  uninterrupted  (baseline)  accuracy  -for  the  primary 
task.  Since  the  time  at  which  this  5— minute  period 
occurred  varied  from  session  to  session,  subjects  were  not 
able  to  anticipate  the  lull  in  warning  activity.  Since  the 
original  constraint  on  the  random  number  generator  was  set 
at  a  5— mi nute  maximum,  expanding  the  longest  lull  to  equal 
five  minutes  changed  the  length  of  each  session  by 
different  amounts,  thus  reducing  even  further  possible 
end-effects.  Including  end  segments,  sessions  were 
designed  to  last  between  30  and  40  minutes,  with  no  two 
sessions  having  the  same  length. 

The  primary  tracking  task  accuracy  was  the  accuracy  of 
the  primary  task  during  each  run,  excluding  the  five  minute 
baseline  period.  The  normalized  primary  task  accuracy  is 
herein  defined  as  the  ratio  of  the  primary  tracking  task 
accuracy  divided  by  the  baseline  accuracy.  This  ratio 
provided  a  measure  of  change  in  performance  caused  by  the 
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addition  o-f  the  secondary  task. 


In-formation  recorded  -for  each  trial  included  the  run 
parameter  code,  the  date  and  time  when  the  test  started, 
and  the  run  parameters  (voice  type,  background  noise,  S/N 
ratio,  and  precursor  format)  used  for  that  particular 
trial.  Primary  and  secondary  task  information  was  recorded 
for  each  subject.  For  the  primary  task,  the  number  of 
stimuli  and  number  of  correct  responses  were  recorded  both 
throughout  each  run  and  for  the  baseline  period.  For  the 
warning  response  task,  each  stimulus  and  its  response  time 
were  recorded.  Responses  for  the  first  three  minutes  and 
last  two  minutes  of  each  test  were  not  recorded.  File 
conventions  are  listed  in  Appendix  Dl.  Subject  and  group 
mean  performance  data  are  listed  in  Appendix  D2  and 
Appendix  D3. 

A  Multiple  Analysis  of  Variance  (MANOVA)  was  used  to 
investigate  the  effects  of  different  voice  types  on  the 
accuracy  and  speed  of  response  to  the  warning  stimuli. 
Before  trying  to  determine  the  effects  of  different  voice 
types,  the  data  were  analysed  to  validate  the  experimental 
design.  Results  :rom  the  validation  steps  helped  determine 
the  final  model  design  for  the  MANOVA.  Statistical 
manipulations  were  performed  using  author-generated 
programs  (see  Appendix  C)  and  subprograms  from  the 
Statistical  Package  for  the  Social  Sciences  (SPSS),  version 
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9  (Nie  et  al . ,  1975;  Hull  &  Nie,  1991). 

.  Design  Validation; 

Step  1 : 

An  author-generated  program  was  used  to  create  new  data 
files  from  the  raw  data  files.  The  four  new  data  files 
created  were  used  as  databases  for  different  measurements; 
one  file  contained  individual  data  points,  one  file  listed 
individual  subject  performance  within  each  run,  one  file 
listed  mean  group  performance  for  each  run,  and  one  file 
combined  individual  data  point,  subject,  and  group  mean 
information. 

Step  2: 

A  summary  of  all  measurements  was  obtained  using  the 
CONDESCRIPTI VE  and  FREQUENCIES  subprograms  of  the  SPSS  to 
validate  that  all  treatments  occurred  the  correct  number  of 
times  and  that  the  design  was  balanced.  Additionally, 
statistics  from  this  summary  were  used  to  validate  the 
results  of  the  author — generated  programs. 

Step  3; 

A  grouped  t-test  for  both  accuracy  and  response  time 
was  performed  within  each  of  the  54  runs.  The  responses  to 
the  first  one-third  warnings  (stimuli  number  1  thru  11) 
were  compared  with  those  of  the  last  one-third  (stimuli 
number  22  through  32) .  These  tests  were  used  to  check  for 
the  presence  of  either  learning  effect  or  fatigue  within 


3  A 


each  run. 


Step  4: 

A  one-way  Analvsis  of  Variance  (ANOVA)  and  regression 
were  per-formed  independently  on  both  accuracy  and  response 
time,  using  the  run  sequence  number  as  the  treatment. 

These  measures  were  used  to  check  -for  the  presence  o-f 
learning  e-f-fect  or  -fatigue  across  the  runs.  Additionally, 
a  regression  of  run  sequence  number  on  the  normalized 

primary  tracking  task  performance  was  calculated  to  supply 
information  on  the  extent  to  which  subjects  acclimated 
themselves  to  the  interjection  of  warnings  over  the  course 
of  the  experiment. 

Effects  on  Accuracy  and  Speed 

The  MANOVA  was  performed  next,  using  accuracy  of 

response  and  sneed  of  response  as  the  criterion  (dependent) 

variables.  Treatments  included  the  voice  types,  precursor, 
background  level,  and  S/N  ratio,  with  blocking  on  the 
individual  subjects.  Two-way,  three-way,  and  four — way 
interactions  of  the  treatments  (except  individual  subjects) 
were  included  in  the  MANOVA  design.  The  run  sequence 
number  and  normalized  primary  task  performance  were 
included  as  covariates  (Hull  S<  Nie,  1931).  The  Null 
Hypothesis  is  that  there  was  no  significant  difference  in 
speed  of  response  or  accuracy  of  response  among  the  several 
treatments.  If  the  Null  Hypothesis  was  not  rejected,  then 
the  conclusion  would  be  that  accuracy  and  speed  of  response 
are  not  affected  by  different  voice  types  in  voice  warning 
systems,  within  the  limitations  of  our  experiment. 


If  the 


Null  Hypothesis  was  rejected,  then  we  could  say  speed 
and/or  accuracy  of  response  were  affected  by  some  or  all  o-f 
the  variables  examined  in  this  experiment. 

A  one-way  ANOVA  was  then  performed  independently  on  the 
accuracy  and  speed  o-f  reaction,  using  those  treatments 
identi-fied  in  the  MANOVA  as  having  a  significant  effect. 
The  ANOVA  employed  the  Scheffe  Method  of  Multiple 
Comparisons  (Neter  §<  Wasserman,  1974)  to  identify  the 
groupings  of  the  treatments.  The  ANOVA  was  also  used  to 
indicate  the  relative  increase  or  decrease  in  performance 
explained  by  the  different  treatments.  Due  to  the 

interaction  of  the  variables,  the  ANOVA  could  not  be 
interpreted  independently,  but  was  only  useful  in  helping 
to  interpret  the  results  of  the  MANOVA. 
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CHAPTER  IV 


AND  ANALYSIS 


This  chapter  is  divided  into  four  sections.  Section 
one  will  describe  the  conduct  of  the  experiment.  Section 
two  will  present  the  data  results.  Section  three  will 


discuss  the 

validation  of 

the 

experi mental 

design,  as 

mentioned  in 

section  four 

of 

Chapter  III. 

Section  four 

will  analyze 

the  effects  of 

the 

treatments  on 

the  accuracy 

and  speed  of  response. 

Conduct  of  the  Experiment 

The  experiment  took  place  in  the  VDCRES  facility 
between  14  June  83  and  6  July  83.  There  were  four  runs 
each  day,  all  in  the  morning.  Subjects  were  given  a 
half-hour  break  between  runs. 

Voice  Warnings 

After  the  recordings  of  professional  announcers  were 
completed,  the  warnings  were  digitized  for  LPC  generation. 
The  mean  length  for  all  thirty-two  warnings  for  the  male 
voice  was  1.1406  seconds,  with  a  standard  deviation  of 
.0538  seconds;  for  the  female  voice  the  mean  length  was 
1.0204  seconds,  with  a  standard  deviation  of  .0485  seconds; 
for  the  machine  voice  the  mean  length  was  1.1262  seconds, 
with  a  standard  deviation  of  .0489  seconds. 
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Training 


Training  and  practice  runs  were  accomplished  two  weeks 
before  the  beginning  o-f  the  experiment.  During  the 
training  and  practice  runs,  the  baseline  tracking  accuracy 
was  used  -for  two  purposes:  setting  the  primary  task 
■frequency  and  eliminating  subjects  with  poor  performance. 
Originally,  the  asterisks  were  generated  on  the  screen  each 
0.7  seconds;  the  mean  reaction  time  to  the  primary  task  was 
about  0.6  seconds,  and  the  mean  primary  task  accuracy  was 
about  97%.  Two  subjects,  whose  performance  -fell 

consistently  below  75%,  were  replaced.  The. primary  task 
interval  was  then  decreased  as  the  subjects'  proficiency 
increased,  and  the  asterisk  was  generated  each  0.6  seconds. 
Mean  primary  task  accuracy  then  -fell,  but  still  remained 
above  90%.  One  subject,  whose  perfomancs  -fell  consistently 
below  75%,  was  replaced.  As  the  subjects  became  more 
pro-ficient,  the  secondary  "warning"  task  was  added  to  their 
training,  using  random  mixes  o-f  the  variables.  Toward  the 
end  o-f  their  training,  the  subjects  underwent  the  same 
conditions  they  would  encounter  in  the  actual  experiment. 
Subjects  were  not  told  when  the  last  "practice  run" 
occurred  and  when  the  -first  "real  run"  began. 

Data  Points 

Fifty— four  runs  were  completed,  representing  the 
balanced  application  of  all  combinations  of  treatments  to 
each  subject.  The  same  ten  subjects  participated  in  all  of 
the  runs.  The  thirty-two  "warning"  stimuli  were  all 
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administered  during  each  session.  Thus  the  number  o-f 

stimuli  recorded  totaled  17,280. 

Data  Transfer 

A  problem  in  data  transfer  occurred  in  that  the  Hewlett 
Packard  (HP)  9845T ,  on  which  the  raw  data  had  been 
recorded,  was  unable  to  transfer  the  data  directly  to  the 
CDC  Cyber  750,  on  which  the  SPSS  MANQVA  program  was 
implemented.  To  overcome  that  problem,  the  authors 
attached  an  Apple  lie  mi crocomputer  to  the  serial  output 
port  of  the  HP.  The  HP  was  then  programmed  to  generate 
line  numbers  and  send  the  data  out  through  its  serial  port, 
thinking  it  was  only  sending  ASCI I-standard  characters  to  a 
printer.  The  Apple  lie  accepted  the  raw  data  and  saved  it 
in  the  form  of  a  BASIC  file.  For  each  run,  the  data  was 
then  transformed  into  a  TEXT  file  within  the  Apple  lie;  in 
that  form  it  could  be  transferred  via  a  MODEM  to  the  CYBER 
computer.  File  conventions  used  in  the  transfer  of  data 
are  outlined  in  Appendix  Dl. 

Results 

For  the  primary  tracking  task,  the  mean  baseline 
accuracy  throughout  all  54  runs  was  89. OX,  with  a  standard 
deviation  of  9.8X.  The  maximum  baseline  accuracy  was 
99.87.,  and  the  minimum  was  21. 7%.  The  mean  normalized 
accuracy  for  all  54  runs  was  96.77.,  with  a  standard 
deviation  of  1.095*4.  The  reduction  in  primary  task 
accuracy  is  statistically  significant  (p=.0013),  indicating 


that  the  primary  task  required  concentration;  the  addition 
o-f  the  secondary  task  affected  the  performance  of  the 
primary  task. 

For  the  secondary  "warning"  task,  the  mean  accuracy 
across  all  54  runs  was  93,5%,  with  a  standard  deviation  of 
2.46%.  The  mean  response  time  (for  correct  responses) 
across  all  54  runs  was  2.909  seconds,  with  a  standard 
deviation  of  0.631  seconds.  The  minimum  response  time  was 
1.331  seconds,  and  the  maximum  response  time  (within  the 
limitations  discussed  in  Chapter  III)  was  5.517  seconds. 

Due  to  the  massive  size  of  the  data  base,  only  mean 
subject  and  group  performances  are  presented  in  the  thesis. 
Except  as  noted,  however,  the  individual  datapoints  were 
used  in  statistical  manipulations  in  order  not  to  lose 
valuable  information.  The  mean  performance  for  each 
subject  within  each  run  is  presented  in  Appendix  D2 
(SUBFIL) .  The  mean  group  performance  for  all  subjects 
within  each  run  is  presented  in  Appendix  D3  (GPFIL) .  An 
explanation  of  the  contents  of  the  preceding  two  files  is 
in  Appendix  D1  (File  Conventions).  Pertinent  information 
about  the  secondary  task  was  extracted  from  the  appendices, 
and  is  presented  in  Table  1;  data  is  listed  in  order  of  run 
sequence  number.  Most  titles  of  Table  1  are  self 
explanatory;  "MN  CORRECT"  is  the  mean  number  of  correct 
responses  (maximum  possible  is  32)  per  subject  for  a 


particular  run. 


Run  # 

V-TYPE 

8ACKGRD 

S/N-RAT 

PRECURS 

MN  CORRECT 

ACCURACY 

RESP-TIME 

1 

1 

I 

1 

1 

1 

Machine 

115 

0 

Voice 

25.700 

0.803 

2.745 

2 

Male 

115 

10 

Tone 

30.500 

0.953 

2.557 

3 

Mai  e 

105 

10 

Tone 

29.800 

0.931 

2.537 

4 

Female 

115 

10 

Repeat 

30.200 

0.944 

2.503 

5 

Male 

115 

5 

Voice 

29.400 

0.919 

2.811 

6 

Machine 

105 

0 

Tone 

28. 100 

0.878 

2.682 

7 

Female 

105 

10 

Voice 

30.700 

0.959 

2.788 

8 

Mai  e 

115 

0 

Repeat 

30.400 

0.950 

2.700 

9 

Machine 

115 

5 

Tone 

27.500 

0.859 

2.762 

10 

Machine 

105 

0 

Voice 

29.600 

0.925 

2.877 

11 

Female 

105 

10 

Repeat 

31.200 

0.975 

2.686 

12 

Machine 

115 

0 

Tone 

29. 100 

0.909 

2.816 

13 

Male 

115 

5 

Tone 

29.800 

0.931 

2.852 

14 

Female 

105 

5 

Repeat 

30.600 

0.956 

2.849 

15 

Female 

105 

0 

Tone 

30.600 

0.956 

2.666 

16 

Male 

115 

0 

Voice 

30.000 

0.937 

2.979 

17 

Female 

115 

5 

Tone 

29.400 

0.919 

2.743 

18 

Male 

105 

0 

Repeat 

31.200 

0.975 

2.958 

19 

Machine 

105 

0 

Repeat 

31.200 

0.975 

3.030 

20 

Female 

115 

0 

Voice 

26.700 

0.834 

3.017 

21 

Machine 

105 

10 

Repeat 

30.800 

0.962 

3.144 

22 

Female 

105  ' 

0 

Voice 

30.000 

0.937 

2.975 

23 

Female 

115 

5 

Voice 

28.700 

0.897 

2.876 

24 

Male 

105 

5 

Tone 

30.700 

0.959 

2.760 

25 

Mai  e 

115 

10 

Repeat 

31.300 

0.978 

3.020 

26 

Machine 

105 

10 

Tone 

29.900 

0.934 

2.722 

27 

Female 

115 

0 

Repeat 

29.400 

0.919 

2.694 

28 

Machine 

115 

10 

Tone 

30.300 

0.947 

2.721 

29 

Machine 

115 

5 

Repeat 

30.800 

0.962 

2.907 

30 

Male 

105 

10 

Repeat 

31.500 

0.984 

2.917 

31 

Female 

115 

0 

Tone 

29.700 

0.928 

2.712 

32 

Male 

115 

5 

Repeat 

31 . 300 

0.978 

2.937 

33 

Male 

105 

10 

Voice 

30.300 

0.947 

2.974 

34 

Female 

105 

10 

Tone 

30.300 

0.947 

2.667 

35 

Machine 

115 

5 

"Voice 

29.400 

0.919 

2.961 

36 

Machine 

115 

10 

Repeat 

30.900 

0.966 

2.973 

37 

Male 

105 

0 

Tone 

29.200 

0.913 

2.693 

38 

Female 

105 

5 

Voice 

30.300 

0.947 

2.788 

39 

Male 

115 

0 

Tone 

28.700 

0.897 

2.661 

40 

Female 

115 

10 

Voice 

29.300 

0.916 

2.806 

41 

Female 

105 

5 

Tone 

29,600 

0.925 

2.583 

42 

Machine 

105 

5 

Tone 

30.000 

0.937 

2.663 

43 

Male 

105 

5 

Voice 

30.400 

0.950 

2.761 

44 

Machine 

115 

10 

Voice 

28.900 

0.903 

2.964 

45 

Female 

115 

5 

Repeat 

29. 100 

0.909 

2.869 

46 

Machine 

105 

10 

Voice 

30.800 

■  0.962 

2.893 

47 

Female 

115 

10 

Tone 

31.000 

0.969 

2.602 

48 

Machine 

115 

0 

Repeat 

30.500 

0.953 

2.764 

49 

Machine 

105 

5 

Repeat 

30.700 

0.959 

2.877 

50 

Machine 

105 

5 

Voice 

30.400 

0.950 

2.953 

51 

Male 

115 

10 

Voice 

30.500 

0.953 

2.857 

52 

Male 

105 

5 

Repeat 

30.900 

0.966 

2,838 

53 

Male 

105 

0 

Voice 

29.000 

0.906 

2.886 

54 

Female 

105 

0 

Repeat 

29.900 

0.934 

2.696 

Table  1.  Mean  Results  by  Run .  Number 
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Validation  of  Experimental  Design 


The  data  were  analyzed  initially  -for  two  purposes. 
First,  statistical  tests  were  performed  to  validate  the 
experimental  design.  Second,  results  from  some  o-f  the 
tests  helped  direct  the  design  for  the  MANOVA  by  providing 
information  on  variables  to  include  as  treatments  and 
covariates.  Throughout  the  analysis,  statisitical 

significance  was  considered  critical  at  the  level  of 
alpha=0. 05. 

Regression  of  Accuracy  Against  Speed 

The  null  hypothesis  was  that  there  was  no  relationship 
between  accuracy  and  speed  of  response.  If  the  null 

hypothesis  could  not  be  rejected,  then  analysis  would  be 
simplified  by  allowing  separate  analysis  of  the  two 
variables. 

Due  to  the  fact  that  the  response  accuracy  in  the  raw 
data  files  is  a  binary  dependent  variable  ("correct"  or 
"incorrect") ,  the  raw  data  file  could  not  be  used  to  check 
for  a  linear  relationship  between  the  accuracy  of  response 
and  the  speed  of  correct  responses  without  extensive 

transf ormati on  (Neter  S<  Wasserman.,  1974).  Using  the  mean 
subject  responses  for  each  >~un,  a  regression  was  tabulated 
for  accuracy  against  speed.  To  investigate  the  effect  of 
voice  type  on  the  relationship,  male  and  female  voices  were 
used  as  "dummy"  variables,  and  the  regression  was  run 

twice:  once  using  accuracy  by  time,  and  again  using  time  by 

accuracy.  The  results  are  in  Table  2.  The  adjusted 
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R-square  indicates  the  percentage  of  the  variation  of  the 


dependent  variable  which  is  explained  by  the  independent 
variable.  The  Slope/Coeff.  column  shows  two  things:  the 
slope  of  the  regression  line  for  accuracy-time,  and  the 
coefficient  for  the  dummy  variables.  The  F/Part-F  column 
shows  the  F-value  for  the  whole  model  on  the  first  line, 
and  the  partial-F  for  the  model  using  stepwise  inclusion  on 
subsequent  lines  (Nie  et  al . ,  1975).  A  significant 
coefficent  for  the  dummy  variable  indicates  that  the  mean 
value  for  the  dependent  variable  is  significantly  different 
for  different  levels  of  the  dummy  variable;  the  slope  of 
the  relationship  is  not  affected.  The  column  indicating 
the  significance  of  F  lists  the  smallest  significance  level 
at  which  the  null  hypothess  can  be  rejected. 


Dependent 

Independent 

Adjusted 

Correl. 

Slope/ 

F  / 

Signif . 

Is  "F* 

Variable 

Variable 

R-Square 

Ccsff . 

Coeff. 

Part-F 

of  F 

Siqnif? 

ACCURACY 

T»HE  OF  RESP. 

0.0199 

-0.1221 

4.651 

0.003 

YES 

TIHE  OF  RESP. 

(Partial) 

-0.0203 

8.133 

0.004 

YES 

HALE  VOICE  (Duaay  Var ) 

0.0169 

4.681 

0.031 

YES 

FEHALE  VOICE  (Duaay  Var) 

0.0015 

0.035 

0.851 

No 

TIME  OF  RESP 

ACCURACY 

0.0199 

-0.1221 

4.480 

0.004 

YES 

ACCURACY  (Partial) 

-0.733 

8.143 

0.004 

YES 

HALE  VOICE  (Duaay  Var) 

-0.033 

0.475 

0.491 

No 

FEHALE  VOICE  (Duaay  Var) 

-0.105 

5.004 

0.026 

YES 

Table  2.  Relationship  Between 
Accuracy  and  Time 


The  negative  slope  of  the  accuracy/t i me  relationship 
indicates  that,  as  accuracy  increased,  the  time  decreased; 
i.e.,  speed  also  increased.  This  finding  goes  against  the 


intuitive  -feeling,  that,  as  speed  increases,  accuracy  should 
decrease ,  since  doing  things  -faster  is  normally 

associated  with  less  care  and  accuracy-  This  -finding  is, 
to  say  the  least,  interesting.  A  possible  explanation  is 
that  those  characteri  sties  o-f  the  warning  (voice  type, 
precursor  -format,  etc.)  which  increased  the  subjects' 
accuracy  also  enabled  them  to  respond  more  quickly  by 
reducing  ambiguity. 

The  statistically  signi-ficant  positive  coe-f -f i ci ent  -for 
the  male— voice  dummy  variable  in  the  -first  regression 
indicates  that  the  mean  accuracy  for  the  male  voice  was 
higher  .  than  that  for  the  female  and  machine  voices. 
Likewise,  the  significant  negative  coefficient  for  the 
female  voice  in  the  second  regression  indicates  that  the 
response  time  for  the  female  voice  was  lower  (i.e.  faster 
reaction)  than  for  the  male  or  machine  voices. 

The  statistical  significance  of  the  interrel  ationship 
between  accuracy  and  speed  emphasized  the  justification  for 
requiring  a  MANOVA  with  two  criterion  variables,  as  opposed 
to  running  two  separate  ANOVA's. 

Within-runs  t-tests 

The  null  hypothesis  was  that  there  was  no  difference  in 
the  mean  accuracy  or  speed  of  response  between  the  first 
one-third  stimuli  within  each  run  and  the  last  one-third 
stimul i . 

An  authoi — generated  program  (Appendix  C2)  was  used  to 
perform  the  grouped  t-tests  of  accuracy  and  response  time 
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within  each  run.  The  results  are  listed  in  Append!;:  El. 
The  was  no  evidence  of  either  -fatigue  or  learning  curve  for 
response  time  within  any  run.  Seven  of  the  fifty-four  runs 
showed  evidence  of  diminished  accuracy  during  the  last 
one-third  responses  when  compared  to  the  first  one-third 
responses.  No  variable  stands  out  as  consistent  among  the 
seven  cases.  Of  the  seven  runs,  two  were  for  the  male 
voice,  two  for  the  female,  and  three  for  the  machine  voice. 
The  cases  were  not  grouped  together  in  sequence  number,  nor 
do  they  appear  predominantly  at  the  beginning,  middle,  or 
end  of  the  experiment.  There  is  thus  no  evidence  that  any 
particular  voice  type  promotes  fatigue  more  than  any  other 
voice  type.  Additionally,  subject  performance  within  each 
run  appears  to  be  fairly  stable. 

ANQVA  and  Regression  of  Accuracy  Across  Runs 

The  null  hypothesis  was  that  the  accuracy  of  response 
remained  constant  across  the  runs;  that  is,  the  subjects' 
accuracy  did  not  show  a  significantly  positive  or  negative 
trend  due  to  acclimation  with  the  experiment. 

The  ANOVA  of  accuracy  across  the  run  numbers  was 
statistically  significant  (F=6.699,  p=0. 000) .  The  Scheffe 
procedure  divided  the  runs  into  two  subsets.  The 

groupings,  however,  were  both  large,  and  not  necessarily 

grouped  near  the  beginning,  middle.  or  end  of  the  run 

numbers.  For  subset  I,  only  runs  11,  18,  19,  25,  30,  32, 

and  47  were  not  included.  All  those  runs  had  the 
"repeated"  precursor  except  run  number  47,  which  had  tone. 
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The  only  run  not  to  be  included  in  subset  II  was  run  number 


1-  Although  the  ANOVA  showed  >  significant  difference  by 
the  treatment  of  run  number,  nothing  became  evidently 
apparent  as  to  the  reason  for  the  difference. 

A  regression  of  accuracy  across  the  runs  did  not  use 
the  raw  data  file,  again  due  to  the  binary  nature  of  the 
"accuracy"  variable  in  that  file  (Neter  S<  Wasssrman ,  1974). 
Therefore,  the  regression  used  the  mean  subject  accuracy 
for  each  run.  Voice  types  were  included  as  dummy  variables 
to  investigate  the  effect  of  voice  type  on  the 
relationship.  The  results  are  in  Table  3. 


Dependent 

Independent 

Adjusted 

Correl. 

Slope/ 

F  / 

Signif . 

Is  "F* 

Variable 

Variable 

R-3quare 

Coeff . 

Coeff. 

Part-F 

of  F 

Signif? 

ACCURACY 

RUN  NUMBER 

0,0160 

0.10074 

3.916 

0.009 

YES 

RUN  NUMBER  (Partial) 

0.001 

5.951 

0.015 

YES 

HALE  VOICE  (Dually  Var) 

0.019 

5.634 

0.018 

YES 

FEHALE  VOICE  (Duamy  Var) 

0.004 

0.294 

0.588 

Ho 

Table  3.  Relationship  Between 
Accuracy  and  Run  Number 


The  regression  indicates  that  the  linear  relationship 
was  statistically  significant.  The  positive  slope  shows 
that  accuracy  increased  (albeit  very  slightly)  over  the 
course  of  the  experiment.  The  significant  positive 
coefficient  for  the  male  voice  indicates  that  the  mean 
accuracy  in  response  to  the  male  voice  was  higher  than  for 
the  female  and  machine  voices.  The  small  value  of 
R-squared  warns  us  that,  although  significant,  the  run 
number  explained  only  a  very  small  percentage  of  the 
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ANOVA  and  Regression  of  Speed  Across  Runs 

The  null  hypothesis  was  that  the  speed  of  response 
remained  constant  across  the  runs;  that  is,  the  subjects' 
time  to  respond  did  not  show  a  significantly  positive  or 
negative  trend  due  to  acclimation  with  the  experiment. 

The  ANOVA  of  response  time  across  runs  was 
statistically  significant  (F=15.853,  p=0.000).  The  Scheffe 
precedure  grouped  the  treatment  effects  into  seven  subsets. 
Each  subset  included  41,  24,  21,  48,  48,  48,  48,  and  37 
runs  respecti vel y.  The  subsets  were  not  obviously  grouped 
near  the  beginning,  middle  or  end  of  the  run  numbers.  No 
single  treatment  was  obvious  in  explaining  the  difference 
between  the  seven  subsets. 

To  investigate  the  possibility  of  a  linear 
rel ati onshi p ,  a  regression  of  the  response  time  with  the 
run  number  was  performed  using  the  raw  data  file.  Only 
response  times  for  correct  responses  were  included.  The 
regression  showed  a  stati stical 1 y  significant  (F=21.831, 
p=0.00)  positive  slope.  However,  the  slope  was  extremely 
small  (0.00149)  and  the  adjusted  R-squared  was  also  small 
‘(0.00129).  To  investigate  further,  another  regression  was 
performed  using  the  subject  mean  responses,  and  including 
voice  types  as  dummy  variables.  The  results  of  this  second 
regression  are  in  Table  4. 
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Dependent 

Independent 

Adjusted 

Correl. 

Slope/ 

F  / 

Bioni f . 

Is  "F3 

Variable 

Variable 

R-Square 

Coeff . 

Coeff . 

Part-F 

of  F 

Sionif? 

TIKE  OF  RESP 

RUN  NUMBER 

0.0060 

0.04581 

2.082 

0.102 

No 

RUN  NUMBER  (Partial) 

0.001 

1.007 

0.316 

No 

KALE  VOICE  (Duaay  Var) 

-0.044 

0.855 

0.355 

No 

FEMALE  VOICE  (Duaay  Var) 

-0.106 

5.054 

0.025 

YES 

Table  4. 

Relat 

ionship 

Between 

Response  Time  and  Run  Number 

The  regression  o-f  speed  across  runs  using  mean  subject 
response  times  was  not  statistically  significant. 

There-fore,  the  null  hypothesis  cannot  be  rejected, 

indicating  that  there  is  little  reason  to  suspect  that  the 
response  time  changed  significantly  as  a  -function  of 
experience.  The  fact  that  the  partial-F  for  the  female 
voice  was  significant  is  merely  an  indication  that, 
although  we  cannot  reject  the  null  hypothesis  that  the 
slope  of  the  regression  is  zero,  the  mean  response  time 
(accurate  responses  only)  for  the  female  voice,  given  that 
all  other  variables  are  already  in  the  model,  is 

significantly  different. 

Regression  of  Normalized  Primary  Task  Accuracy  Across 
Runs 

The  regression  of  the  normalized  primary  task  accuracy 
(as  defined  in  section  four  of  Chapter  III)  across  runs  was 
an  indication  of  the  extent  to  which  the  subjects  were 
distracted  by  the  secondary  task.  If  the  ratio  was  to 
increase  as  the  run  number  increased,  then  that  would 
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indicate  that  the  subject  was  becoming  less  distracted  bv 
the  warnings  as  the  experiment  progressed-  The  null 
hypothesis  was  that  normalized  accuracy  was  constant  across 
the  runs;  that  is,  the  subjects  did  not  become  more  or  less 
bothered  by  the  warnings  as  time  went  on.  The  results  of 
the  regression  are  in  Table  5. 


Dependent 

Independent 

Adjusted 

Correl. 

Slope/ 

F  / 

Sionif. 

Is  "F" 

Variable 

Variable 

R-Square 

Coeff . 

Coeff. 

Part-F 

of  F 

Signif? 

NORM  ACC 

RUN  NUMBER 

0.0126 

0.10106 

3.293 

0.020 

YES 

RUN  NUMBER  (Pa 

I'tial) 

0.001 

5.822 

0.016 

YES 

KALE  VOICE  (Dusay  Var) 

0.009 

0.558 

0.455 

No 

FEMALE  VOICE  (Duaay  Var) 

0.024 

4.198 

0.041 

YES 

Table  5.  Relationship  Between 
Normalized  Primary  Task  Accuracy  and  Run  Number 


The  results  of  the  regression  show  a  small  positive 
relationship  between  the  run  number  and  the  normalized 
primary  task  accuracy.  This  may  be  an  indication  that,  as 
the  subjects  became  more  accustomed  to  the  experiment,  they 
were  less  distracted  from  their  primary  task.  Again, 
however,  both  the  slope  and  the  adjusted  R-souared  are 
extremely  small,  indicating  that,  although  significant,  the 
relationship  is  slight. 

The  results  of  the  analyses  for  accuracy,  speed,  and 
normalized  primary  task  performance  across  run  numbers 
showed  slight  but  significant  amounts  of  the  variation 
explained  by  the  run  number  (i.e.,  how  far  along  the 
experiment  had  progressed).  The  fact  that  the  amounts 
explained  were  slight  suggests  that  the  subjects  were  at  or 


near  the  top  o-f  the  learning  curve,  and  were  fairly 
proficient  in  the  tasks  assigned  to  them.  With  regard  to 
response  time,  although  the  ANOVA  indicated  significant 
subgroups  for  the  treatment  effects,  the  regression  failed 
to  indicate  any  statistically  significant  linear 
relationship.  The  fact  that  the  amounts  explained  were 
significant  in  the  ANOVA  was  cause  for  including  the  run 
number  and  normalized  primary  task  performance  as 
covariatss  in  the  MANOVA  design. 

Analysis  of  Treatment  Effects 


MANOVA 

A  MANOVA  was  performed,  using  voice  type,  background 
noise,  signal  to  noise  ratio,  and  precursor  type  as  the 
treatments,  with  blocking  on  the  subject  number.  Subject 
number  was  not  included  in  any  of  the  two,  three,  or  four 
way  interaction  effects.  The  primary  reason  for  blocking 
on  the  subject  number  was  to  explain  as  much  of  the 
variation  as  possible  due  to  the  obviously  different 
performance  norms  for  each  subject  (see  Appendix  D2) .  The 
three  way  interaction  of  background  noise  by  signal  to 
noise  ratio  by  precursor  was  not  included  in  the  analysis 
because  the  CYBER  computer  kept  getting  a  large  number 
which  it  was  incapable  of  handling  when  that  combination 
was  part  of  the  MANOVA  design.  The  MANOVA  was  first 
examined  using  Hotelings's  T-squared  to  test  the  null 
hypothesis  that,  in  this  repeated-measures  design,  the 
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vari ancs-covari ance  matrix  between  treatments  was 
homogeneous.  If  the  homegeneity  assumptions  were  met,  the 
Hotel  lings'  T=  and  the  usual  F  test  give  identical 
results  (Hull  S<  Nie,  1981).  The  results  o-f  the  MANGVA  are 
in  Appendix  E2.  A  summary  o-f  the  significant  effects  has 
been  brought  f^ward  into  the  text,  and  is  in  Table  6, 
below.  After  -amining  the  multivariate  tests  for 

si gni f i cance ,  the  univariate  F-statistics  were  examined  to 
see 


if  the  groups  are  significantly  separated  by 
any  of  the  variables  when  considered  one  at  a 
time.  We  might  discover  that  a  single  variable 
is  quite  powerful  in  separating  groups,  and  when 
this  is  the  case  we'll  have  found  a  simpler 
explanation  for  the  group  differences  (McNichols, 

1980,  p.  7-69). 

Some  of  the  treatments  may  have  a  significant  effect  on 
the  combined  effects  of  accuracy  and  speed,  but  may  not  be 
significant  on  one  or  the  other  by  itself.  Alternatively, 
a  treatment  may  not  be  significant  when  contemplating  the 
combination  of  accuracy  and  speed,  but  may  be  significant 
in  explaining  the  differences  of  one  without  the  other.  In 
the  latter  case,  the  word  "YES"  was  bordered  by  parentheses 


in  Appendix  E2  and  in  Table  6. 


TREATMENT 

MULTIVARIATE 

SIGNIFICANCE 

UNIVARIATE 

SIGNIFICANCE 

ACCURACY  RESPONSE  TIME 

VOICE  TYPE  CVTYPE3 

YES 

YES 

YES 

BACKGROUND  CBKGRD1 

YES 

YES 

No 

SIG/NOISE  CSNRAT1 

YES 

YES 

No 

PRECURSOR  tPRECl 

YES 

YES 

YES 

SUBJECT 

NO. 

YES 

YES 

YES 

VTYPE  X 

BKGRD 

YES 

YES 

YES 

VTYPE  X 

SNRAT 

No 

Nd 

(YES) 

VTYPE  X 

PREC 

YES 

No 

YES 

BKGRD  X 

SNRAT 

No 

No 

(YES) 

BKGRD  X 

PREC 

YES 

YES 

YES 

SNRAT  X 

PREC 

YES 

No 

YES 

VTYPE  X 

BKGRD  X  SNRAT 

No 

No 

No 

VTYPE  X 

BKGRD  X  PREC 

YES 

YES 

No 

VTYPE  X 

SNRAT  X  PREC 

YES 

YES 

YES 

VTYPE  X 
SNRAT 

BKGRD  X 

X  PREC 

YES 

YES 

No 

Table  6. 

Significant  MAN OVA 

Effects 

0-f  the  -fifteen  treatments  and  interactions  listed,  all 
but  three  di scriminated  ’  in  the  multivariate  sense.  The 
single-treatment  effects  of  voice  type,  precursor,  and 
subject  number  discriminated  in  the  multivariate  and  in 
both  univariate  analyses. 

There  were  only  three  multiple-effect  interactions  to 
discriminate  in  all  three  analyses:  the  two-way  interaction 
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of  voice  type  by  background,  the  two-way  interaction  of 
background  by  precursor,  and  the  three-way  interaction  of 
voice  type  by  signal  to  noise  ratio  by  precursor. 

Interestingly,  two  of  the  interactions  (voicetype  by 
signal  to  noise  ratio  and  background  by  signal  to  noise 
ratio)  were  effective  in  di scriminating  on  response  time 
for  the  univariate  analysis;  this  di scrimination  was  lost, 
however,  when  looking  at  the  multivariate  effect  on 
accuracy  and  response  time. 

Investigation  of  Significant  Treatment  Effects 

Because  all  single-treatment  effects  were  significant 
in  the  multivariate  sense  for  accuracy  and  speed,  a 
univariate  ANOVA  was  run  for  each  of  the  variables 
individually.  The  detailed  results  of  the  ANOVAs  appear  in 


Appendix  E3. 

A  Summary 

of  the  results 

appears  below 

Table  7. 

UNIVARIATE 

SIGNIFICANCE 

TREATMENT 

ACCURACY 

RESPONSE  TIME 

VOICE  TYPE 

[ VTYPE3 

YES 

YES 

BACKGROUND 

CBKGRD3 

YES 

No 

SIG/NOISE 

C  SNR AT  3 

YES 

No 

PRECURSOR 

CPREC3 

YES 

YES 

Table  7. 
Uni  variate 

Significant 
ANOVA  Effects 
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The  results  of  the  univariate  ANOVAs  using  the  complete 
database  agree  with  the  univariate  single-treatments  or  the 
MANOVA  using  the  mean  subject  performance  data.  The  voice 
type  and  precursor  were  significant  for  both  accuracy  and 
response  time,  while  background  noise  and  signal  to  noise 
ratio  were  significant  only  for  accuracy. 

The  Scheffe  Method  of  Multiple  Comparison  was  again 
used  to  identify  the  groupings  of  the  treatments.  Detailed 
results  are  in  Appendix  E3,  and  are  summarised  below.  In 
the  following  description,  the  word  "better"  will  appl^  to 
statistically  significant  greater  accuracy  or 
shorter  response  times.  By  variable,  the  results  were 
as  follows: 

Voice  Type 

Accuracy  for  the  male  voice  was  better  than  for 
the  female  or  machine.  Response  time  for  the  female 
voice  was  better  than  for  the  male  or  machine; 

response  time  for  the  male  voice  was  better  than  for 
the  machine. 

Background 

Accuracy  at  105  db  was  better  than  at  115  db. 
Response  time  was  not  statistically  differentiated  by 
differences  in  the  background  engine  noise. 

Signal  to  Noise  Ratio 

Accuracy  for  warnings  presented  at  10  db  above  the 
headphone  conversation  level  was  better  than  other 
signal  to  noise  ratios;  accuracy  for  warnings 

presented  at  5  db  was  also  better  than  at  0  db. 
Response  time  was  not  affected  by  the  different  signal 
to  noise  ratios. 

Precursor 

Accuracy  for  the  repeated  warning  (double 
call-out) ,  in  which  the  warning  acted  as  its  own 
precursor,  was  better  than  the  other  two  presentation 
styles.  Response  time  for  the  tone  precursor  was 
better  than  for  repeated  or  voice  precursors. 


When  trying  to  make  sense  of  the  results  of  these 


analyses,  two  things  must  be  kept  in  mind: 


1.  The  results  o-f  the  ANOVA  should  not  be 
explained  by  themselves,  but  must  be  interpreted  in 
light  o-f  the  results  of  the  MANOVA. 

2.  The  response  time  was  calculated  only  for 
those  responses  which  were  correct.  Response  times 
for  innacurate  responses  were  not  included. 
Therefore,  a  "better"  response  time  means  that  the 
response  was  faster,  given  the  condition  that  the 
response  was  correct  in  the  first  place. 

Differences  by  Subject's  Sex 


Because  subject  number  was  significant  in  the 
multivariate  sense,  an  obvious  question  arose:  was  there 
any  explainable  aspect  of  the  subjects  which  could  account 
for  this  signif icance?  To  investigate  that  question,  an 
ANOVA  of  accuracy  and  response  by  sex  of  the  subject  was 
in  addition  to  an  analysis  of  the  above 
variables.  An  analysis  for  grouping  was 

performed  only  on  the  basis  of  the  subject's  sex  for  two 
reasons: 

(1)  The  authors  anticipated  a  question  as  to  whether 
men  and  women  responded  differently. 


perf ormed 
moderating 


(2)  Because  this  was  not  intended  to  be  a  demographic 
study,  the  authors  limited  the  analysis  to  one  of  the 
more  obvious  differences  among  the  subjects.  Since 
there  were  only  ten  subjects,  the  authors  felt  that 
any  deeper  analysis  of  performance  by  the  subject's 
sex  would  be  fraught  with  misinterpretation. 


The  detailed  results  of  the  analysis  are  in 
Appendix  E4.  A  summary  of  the  results  reveals  the 
f ol lowing: 

The  subject's  sex  was  significant  for  both 

accuracy  and  response  time. 
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The  male  subjects  performed  better  -for  accuracy 
than  the  -females;  the  -females  per-formed  better  -for 
response  time. 

The  male  subjects  showed  no  statistical  di-f-ference 
in  their  accuracy  -for  any  o-f  the  three  voice  types. 
With  respect  to  speed,  the  male  subjects  showed  no 
statistical  di-f-ference  in  response  time  between  the 
male  and  -female  voice  (subset  1  in  the  Sche-f-fe  method) 
or  between  male  and  machine  voice  (subset  2);  however, 
they  did  respond  better  in  subset  1  than  in  subset  2. 

The  female  subjects  had  better  accuracy  in  response 
to  the  male  voice  type  than  to  the  female  or  machine. 
They  had  a  better  response  time  to  the  female  voice 
than  to  the  male  or  machine  voices. 

When  evaluating  the  subject's  perfurmance  based  on  sex, 
it  should  be  noted  that  both  the  primary  tracking  task  and 
the  secondary  warning  task  were  completely  new  tasks  for 
all  subjects,  and  that  all  subjects  received  equal  amounts 
of  training. 

Length  of  Warnings 

The  mean  length  of  the  warnings  given  by  the  female  voice 
was  significantly  shorter  than  that  given  by  both  the  male 
voice  and  the  machine  voice;  that  is,  it  took  the  female 
speaker  less  time  to  call  out  the  warning.  The  length  of 
the  three-word  warning  (right  red  one,  etc.)  as  spoken  by 
the  female  voice  was  approximately  0. 12  seconds  shorter 
than  that  of  the  male  or  machine  voice.  The  difference  in 
the  speed  of  response  to  the  different  voice  types  was 
shown  to  be  significantly  different,  with  the  response  to 
the  female  voice  approximately  0.07  seconds  faster  than  to 
the  male  or  machine  voice.  This  raised  the  question  of 


whether  the  faster  response  time  was  due  to  the  shorter 


length  of  the  warning  itself 

To  investigate  this  possibility,  the  response  times 
were  adjusted  by  the  mean  length  of  the  warning.  The 
length  of  each  warning  was  dependent  on  the  voice  type,  the 
length  of  the  precursor,  and  the  length  of  the  three-word 
warning  phrase.  Additionally,  a  short  pause  was 
interjected  between  the  precursor  and  the  warning  itself  to 
prevent  forward  masking.  The  duration  of  each  warning  is 
listed  in  Table  8. 


REPEATED  WARNING 

VOICE  PRECURSOR 

TONE  PRECURSOR 

MALE  VOICE 

2.267 

1.641 

1.421 

FEMALE  VOICE 

2.024 

1.590 

1.298 

MACHINE  VOICE 

2.238 

1.626 

1.407 

Table  8.  Length  of  i ‘darnings,  in  Seconds 


Another  ANOVA  was  run,  after  the  appropriate  time  was 
subtracted  from  the  actual  response  time  for  each  response. 
The  results  of  the  ANOVA  using  adjusted  response  times  are 
in  Appendix  E5.  A  summary  of  the  significant  effects  are 
in  Table  9. 
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UNIVARIATE 

SIGNIFICANCE 

TREATMENT 

ACCURACY 

RESPONSE  TIME 

VOICE  TYPE 

C VTYPE ] 

YES 

YES 

BACKGROUND 

CBKGRD3 

YES 

No 

SIG/NQISE 

C  SNR AT  3 

YES 

No 

PRECURSOR 

CPREC3 

YES 

YES 

Table  9.  Significant  Un ivar iate  ANQVA  Effects 
Using  Adjusted  Response  Times 


As  can  be  seen  by  comparing  Table  9  with  the  results 
using  unadjusted  response  times,  those  variables  which  were 
significant  using  normal  response  time  were  still 
significant  discriminators  when  using  the  adjusted  response 
time.  As  expected,  the  accuracy  results  did  not  change. 
The  results  for  response  time,  however,  did  change.  By 
variable,  the  changes  were  as  follows: 


Voice  Type 

After  adjusting  the  response  times  to  remove  the 
length  of  the  warning,  the  male  voice  had  a  better 
response  time  than  either  the  female  or  machine. 
There  was  no  significant  difference  between  the 
response  times  for  the  female  and  machine  voices. 

Background 

Background  engine  noise  did  not  statistically 
distinguish  between  response  times,  just  as  with  the 
'unadjusted  times. 

Signal  to  Noise  Ratio 

Response  time  was  still  not  affected  by  the 
different  signal  to  noise  ratios. 

Precursor 

Using  the  adjusted  response  times,  the  repeated 
warning  (double  call-out)  was  not  only  more  accurate, 
but  also  had  a  shorter  response  time.  Since  the 
adjusted  response  time  is  a  measure  of  the  time  from 


the  end  of  the  entire  warning,  the  significantly 
shorter  adjusted  response  time  for  the  double  call-out 
is  not  surprising.  The  voice  precursor,  using  the 
word  "warning",  was  associated  with  a  better  response 
time  than  the  tone  precursor,  although  there  was  no 
distinction  betwen  these  latter  two  with  respect  to 
accuracy. 

Subject  Sex 

Women  still  had  a  signif icantly  better  response 
time  for  correct  responses.  Among  both  the  male  and 
female  subjects,  the  grouping  of  the  subsets  by  voice 
type  changed  after  adjusting  for  the  length  of  the 
warnings. 

For  both  subject  sexes,  adjusted  response  time  for 
the  male  voice  type  was  si gni f i cantl y  better  than  for 
female  or  machine.  The  adjusted  response  times  for 
female  and  machine  voice  types  were  not  significantly 
different  from  each  other. 


Adjusting  the  response  times  for  the  length  of  the 
warning  indicates  that  the  male  voice  may  be  better. 
Response  time  is  normally  measured  from  the  beginning  of 
the  warning,  however,  not  the  end.  Therefore,  these 
results  should  be  interpreted  with  caution,  and  only  in 
light  of  the  interaction  of  effects  highlighted  by  the 
MANOVA. 


Subject  Questionnaire 

To  help  ascertain,  in  retrospect,  the  impressions  of  the 
subjects  themselves  regarding  the  experiment,  a  short 
questionnaire  was  completed  by  each  subject  after  the  last 
day  of  the  experiment.  A  copy  of  the  questionnaire,  with 
the  instructions  for  its  completion,  is  in  Appendix  F.  The 
subject  responses  have  been  annotated  on  the 
questionnnaire.  Analysis  of  the  completed  questionnaires 
reveals  the  following: 
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1.  Six  out  of  the  ten  subjects  (all  -four  males, 

two  o-f  the  -females)  -felt  more  comfortable 

responding  to  the  MALE  voice,  while  -four  of  the 
subjects  (ail  female)  felt  more  comfortable 

responding  to  the  FEMALE  voice.  ■  None  chose  the 
MACHINE  voice. 

2.  Six  of  the  subjects  (all  four  males,  two  of 
the  females)  felt  they  responded  more  accurately 

to  the  MALE  voice,  while  four  (all  female)  felt  they 
responded  more  accurately  to  the  FEMALE  voice. 

None  chose  the  MACHINE  voice. 

3.  Four  of  the  subjects  (one  male,  three  females) 

felt  they  responded  more  quickly  to  the  MALE 

voice,  while  six  subjects  (three  males,  three  females) 
felt  they  responded  more  quickly  to  the  FEMALE 

voice.  None  chose  the  MACHINE  voice. 

4.  Four  of  the  subjects  (two  males,  two  females) 
preferred  the  MALE  voice,  while  six  (two  males, 
four  females)  preferred  the  FEMALE  voice.  None 
chose  the  MACHINE  voice. 


Especially  because  the  subjects  involved  were  not  rated 
crewmembers,  drawing  inferrences  from  these  responses  is 
dangerous.  Two  points,  however,  stand  out: 

1.  None  of  the  subjects  preferred  the  machine 
voice  nor  felt  they  responded  best  to  that  voice  type. 

2.  Considering  the  number  of  subjects,  there  was 
a  fairly  even  split  for  choosing  between  the  male  and 
female  voice  for  comfort,  response  and  preference. 
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CHAPTER  V 


CCNCLUS I ONS 

A  basic  philosophy  throughout  this  experiment  was  that 
a  response  must  be  accurate  to  be  useful.  Per  that  reason, 
the  authors  restricted  the  discussion  of  response  time  to 
correct  responses. 

The  strong  linear  relationship  between  accuracy  and 
speed  of  response  indicates  that  any  interpretati on  of  the 
effects  of  a  variable  on  accuracy  or  speed  cannot  be 
interpreted  independently;  an  assessment  of  what  is  “best" 
can  only  be  made  in  light  of  its  significance  in  the 
multivariate  sense.  Each  of  the  variables  examined  (voice 
type,  backgound  engine  noise,  signal  to  noise  ratio  over 
the  headphones,  and  precursor)  was  significant  in  its 
effect  on  the  combined  criterion  of  accuracy  and  speed. 
Additionally,  interactions  among  the  independent  variables 
themselves  were  significant  in  their  effect,  down  to  the 
four — way  level  of  interaction. 

The  results  of  the  experiment  show  that  there  is  a 
statistically  significant  difference  in  response  to 
warnings  given  by  different  voice  types.  The  male  voice 
was  associated  with  a  greater  accuracy  than  that  of  the 
female  or  machine  voice.  For  time  of  response,  the 
findings  are  slightly  more  complicated.  The  unadjusted 
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results  point  to  the  -female  voice  as  that  which  was 
associated  with  -faster  response  times,  given  that  the 
responses  were  accurate.  After  adjusting  for  the  length  of 
the  warnings,  however,  the  male  voice  was  associated  with 
the  faster  response.  The  machine  voice  was  associated  with 
the  least  accuracy  and  slowest  speeds. 

Within  the  range  of  background  engine  noises  examined 
(105  and  115  db) ,  the  lower  engine  noise  was  associated 
with  greater  accuracy.  There  was  no  significant 
distinction  for  the  time  of  response. 

For  the  signal  to  noise  ratio,  in  which  the  loudness  of 
the  voice  warning  was  measured  against  the  normal 
conversation  level  over  the  headset,  the  highest  ratio  (10 
db)  was  associated  with  the  most  accurate  response.  The 
lowest  ratio  (0  db) ,  in  which  the  warning  was  given  at  the 
same  volume  as  the  conversat i on ,  was  associated  with  the 
least  accuracy.  There  was  no  significant  difference  in 
response  time  among  the  three  levels  (0,  5,  S<  10  db) 
ex ami ned. 

The  precursor  variable  examined  three  delivery  formats: 
tone  precursor,  voice  precursor,  and  repeated  warning.  The 
repeated  warning,  wherein  the  warning  acted  as  its  own 
attention-getting  device,  was  associated  with  the  highest 
level  of  accuracy.  For  response  times,  the  tone  precursor 
was  associated  with  the  fastest  response,  given  that  the 
responses  were  accurate.  After  adjusting  for  the  length  of 
the  warning,  however,  the  repeated  warning  was  associated 
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with  the  -faster  response  times  as  well  the  greater 
accuracy . 

Although  this  experiment  was  not  intended  to  be  a 
demographic  study,  the  results  prompted  some  limited 
investigation  o-f  di -f-f  erences  in  accuracy  and  speed 
associated  with  the  sex  of  the  subject.  The  subject  sex 
was  significant  in  both  univariate  analyses.  The  men  were 
associated  with  the  greater  accuracy,  while  women  were 
associated  with  the  faster  time,  given  an  accurate 
response.  For  the  men,  voice  type  did  not  discriminate  in 
the  accuracy  of  response.  The  women  responded  more 
accurately  to  the  male  voice. 

The  significance  of  voice  type  on  the  speed  of  response 
is  slightly  more  complicated  when  grouping  the  subjects  by 
sex.  For  the  men,  the  unadjusted  times  indicated  no 
significant  difference  in  response  time  between  the  female 
and  male  voice  types;  the  adjusted  times,  however,  showed  a 
significant  difference  in  response  time  between  the  male 
and  female  voice  types  <male  being  better).  Among  women, 
response  times  for  the  unadjusted  times  clearly  pointed  to 
the  female  voice  for  the  fastest  response  times.  For 
adjusted  times,  conversely,  the  indications  were  just  as 
clear,  but  in  favor  of  the  male  voice  for  the  fastest 
response  times. 
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CHAPTER  VI 


DISCUSSION  ftlxJlD 
RECOMMENDAT I OMS 


The  results  of  this  experiment  contradict  the  common 
belie-f  that  the  -female  voice  is  "better"  in  a  cockpit  voice 
warning  system.  The  higher  accuracy  was  associated  with 
the  male  voice.  While  the  shorter  response  time  was 
associated  with  the  -female  voice,  this  latter  association 
is  tempered  by  two  -facts: 

1.  Response  times  were  calculated  for-  correct 
responses  only.  Thus,  the  shorter  time  is  dependent 
on  the  prior  condition  of  accuracy,  which  was 
associated  with  the  male  voice. 

2.  There  was  a  significant  difference  in  the 
length  of  the  warnings  given  by  the  male  and  female 
voices.  When  compensation  was  made  for  the  length  of 
the  warnings,  the  male  voice  was  more  significantly 
associated  with  shorter  response  times. 

Because  response  time  is  normally  measured  from  the 
onset  of  the  emergency,  i.e.  the  beginning  of  the  warning, 
adjusting  the  response  time  for  the  length  of  the  warning 
message  must  be  treated  with  caution. 

Simpson  and  Williams  <1980)  found  that  response  times 
to  "semantic  context"  warnings  were  faster  than  those  to 
warnings  with  a  tone  precursor,  despite  the  fact  that  the 
tone  took  less  time  than  the  semantic  context  addition. 
They  expected  the  word  "warning"  would  act  in  the  same 
manner  as  the  semantic  context.  By  their  expectations,  the 


voice  precursor  should  have  been  associated  with  the 
shortest  response  time.  Contrarily,  the  response  time  tor 
the  tone  precursor  was  signiticantly  shorter  than  tor  the 
voice  precursor  when  using  unadjusted  response  times.  The 
word  "warning"  apparently  did  not  act  in  exactly  the  same 
manner  as  changing  the  semantic  context,  as  Simpson  anc 
Williams  expected. 

Adjusting  the  response  time  tor  the  length  or  the 
warnings  also  showed  some  interesting  aspects  ot  the 
warning  tormat.  The  repeated  warning  tormat  was  associated 
with  the  tastest  response  time  atter  adjusting  the  response 
times  tor  the  warning  length.  This  result  is  hardly 
surprising  tor  two  reasons:  (1)  a  larger  number  was 
subtracted  trom  the  total  response  time  due  to  the  longer 
message  tormat,  and  (2)  the  repeated  warning  precursor 
provides  more  intormation  than  either  a  tone  or  the  word 
"warning " . 

What  is  interesting,  however,  was  the  ettect  on  the 
distinction  between  the  other  two  precursor  tormats,  tone 
and  the  voice  "warning".  Without  adjustment,  the  tone  was 
associated  with  the  -faster  response  time  for  correct 
responses.  After  adjustment,  however,  the  voice  precursor 
was  associated  with  a  faster  response  than  the  tone,  and 
the  tone  precursor  was  associated  with  the  slowest  response 
time. 

Although  the  better  accuracy  was  associated  with  the 
higher  signal  to  noise  ratio,  this  result  should  not  be 
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extended  ad  infinitum’.  Above  a  certain  level,  sound  can  be 
not  only  irritating  but  physically  harm-ful.  The  signal  to 
noise  ratios  used  in  this  experiment  were  sufficient  to 
deliver  the  warning  without  apparent  irritation  or  harm. 
Nonetheless,  further  testing  may  be  needed  to  ascertain  the 
optimal  level. 

All  variables  examined  were  significant  in  the 
multivariate  sense,  but  not  necessarily  in  the  univariate 
sense  for  both  accracy  and  time.  Some  of  the  interactions 
(down  to  the  four-way  level)  were  also  significant. 
Because  not  only  the  variables,  but  also  the  interaction  of 
the  variables,  were  significant  in  many  instances,  it  is 
recommended  that  further  study  be  conducted  on  the 
interaction  of  the  variables. 

Future  experiments  should  investigate  what  aspects  of 
the  voice  are  most  signif icantly  associated  with  accuracy 
and  speed  of  response.  These  might  include,  but  are  not 
limited  to:  frequency,  intensity,  pitch,  timbre, 

intonation,  and  delivery  style. 

Because  the  results  of  this  experiment  contradict 
previous  experiments  and  studies,  the  authors  recommend 
that  additional  empirical  evidence  be  gathered.  Future 
experiments  should  include  crew  members  of  both  sexes  and 
should  be  conducted  in  aircraft  simulators  and/or  actual 
aircraft.  Voice  recordings  should  be  made  by  persons 
familiar  with  air  traffic  control  vocabulary  and 
procedures.  Professional  speakers  with  air  traffic  control 


experience  would  be  preferab1'-. 

Giv^n  that  the  results  o-f  this  experiment  hold  with 
replication,  the  authors  recommend  the  use  o-f  a  male  voice 
in  a  cockpit  voice  warning  system  -for  military  aircra-ft. 
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APPENDIX  *=* 
VOCRES  FACILITIES 
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INTRODUCTION 


Air  and  ground  crew  voice  communications  may  be  degraded  by  a  variety  of  system  and 
environmental  factors  that  include  electrical  or  acoustical  noise  or  both,  radio 
interference,  jamming,  communication  signal  processing  and  various  other  factors  that 
prohibit  effective  communication.  Vigorous  research  activities  must  be  maintained  to 
identify  and  quantify  elements  that  cause  suqh  deterioration  and  to  develop  principles, 
techniques  and  guidelines  that  will  minimize  adverse  effects  and  optimize  voice 
communications.  Analytical  studies  of  communication  system  performance,  environmental 
influences  and  the  man-in-the-loop  element  must  be  carried  out  under  carefully  controlled 
conditions  that  simulate  to  the  greatest  extent  possible,  the  practical,  operational 
situations  of  concern.  Such  efforts  are  possible  in  controlled  laboratory  environments 
where  special  instrumentation  can  be  used  to  create  the  essential  elements  of  human 
factors  and  communication  system  networks  being  investigated. 

A  Voice  Communication  Research  and  Evaluation  System  (VOCRES),  located  in  the 
Biodynamics  and  Bioengineering  Division  of  the  Air  Force  Aerospace  Medical  Research 
Laboratory  has  been  developed  to  provide  the  capability  for  comprehensive  research,  test 
and  evaluation  activities  in  the  voice  communications  effectiveness  arena.  VOCRES,  the 
subject  of  this  report,  has  been  designed  to  replicate  system  and  environmental  variables 
believed  to  have  significant  influence  on  operational  communication.  Using  VOCRES, 
various  elements  of  voice  communications  can  be  analyzed  either  individually  or  in 
component  clusters.  Using  this  method  of  analysis,  problem  areas  can  be  identified, 
attacked  and  the  overall  operation  enhanced.  The  effectiveness  of  various  interfering 
signals  may  also  be  evaluated  by  their  insertion  into  the  communication  system.  The 
operational  procedures  and  materials  used  in  the  laboratory  are  well  standardized  and 
provide  data  with  a  high  degree  of  reliability.  ■ 

This  report  describes  the  VOCRES  system  instrumentation  in  some  detail  as  well  as  the 
psychoacoustical  procedures  used  in  the  overall  operation  of  the  voice  communication 
research  program.  The  key  element  of  the  overall  program  is  VOCRES.  Other  component 
systems  are-  essential  to  the  realistic  replication  of  communication  situations  for 
expanding  the  technology  base  as  well  as  performing  discrete  measurements  required  for 
the  treatment  of  specific  problems. 


APPROACH 

The  general  approach  employed  in  this  program  involves  the  participation  of  volunteers 
who  communicate  as  talkers  and  listeners  under  controlled  conditions  that  replicate  the 
specific  communication  environments  being  .evaluated.  Subjects  are  stationed  at  custom- 
designed  consoles  and  communicate  with  standardized  or  special  purpose  (speech) 
vocabulary  materials  while  various  system  and  environmental  characteristics  or 
equipment  are  varied  and  the  resulting  communication  effectiveness  is  quantified. 
Elements  commonly  varied  are  microphones,  earphones,  ambient  noise  level  at  the  crew 
station,  helmets  and  oxygen  masks,  aircraft  radios,  jamming  signal  type  and  modulation, 
jammer  to  signal  power  ratios,  and  receiver  input  power.  Data  derived  from  these  efforts 
may  be  used  to  establish  baseline  communication  system  performance  profiles,  for 
comparative  testing  of  specific  communication  system  components,  such  as  radios, 
intercoms,  microphones,  earphones,  and  voice  processors.  The  data  are  also  used  to 


quantify  the  performance  of  a  specific  component  in  a  specific  environment.  Subjective 
comments  from  active  aircrew  personnel  who  have  experienced  the  VOCRES  reveal  that 
the  validity  of  the  system  and  of  the  approach  is  quite  good. 


INSTRUMENTATION 


VOCRES:  General  System 

The  VOCRES  system  is  an  aggregate  of  four  different  subsystems  integrated  into  a  voice 
communication  netwoik  that  includes  ten  individual  communication  stations  and  one 
control  station.  The  individual  communication  consoles  are  located  in  a  large 
reverberation  chamber  and  the  master  console  is  located  in  a  control  room  adjacent  to 
the  chamber.  The  general  physical  assemblage  of  the  individual  subsystems  and  the 
integrated  system  is  displayed  in  Figure  1. 
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Flgura  1.  SUBSYSTEMS  INTEGRATION 

The  subsystems  include  (1)  an  AIC-25  aircraft  intercommunication  system  (11  stations), 
for  use  with  Air  Force  standard  communications  headgear,  (2)  an  air  respiration  system 
with  A-19  diluter-dernand  regulators  for  use  with  standard  oxygen  masks,  (3)  a  high 
intensity  sound  source  for  duplicating  operational  acoustical  environments  occupied  by 
crew  members  and  (4)  a  central  processing  unit  that  controls  all  stations  and  conducts  the 
individual  testing  sessions  and  conditions,  i.e„  presents  materials,  monitors  participant 
activity,  records,  stores  and  analyzes  responses,  and  provides  analyzed  data  in  tabular  or 
graphic  form  or  both.  The  overall  system  is  adaptable  to  the  incorporation  of  various 
aircraft  radios,  communication  jammers,  and  the  like,  that  are  not  integral  components  of 
VOCRES. 

Each  of  the  ten  communication  consoles  or  stations  is  equipped  with  an  AIC-25 
intercommunication  terminal,  an  A- 19  respiration  terminal,  a  display/subject  response 
unit,  a  keyboard  for  communication  performance  task  response  from  the  participants  and 


a  large  volume  unit  (VU)  meter  that  indicates  voice  level  of  communications  generated  at 
that  station  (see  Figure  2).  The  system  can  be  operated  with  any  number  of  one  to  ten 
volunteers.  The  psychophysical  paradigm  used  most  often  is  a  "round  robin"  procedure 
where  each  subject,  in  turn,  performs  as  a  talker  while  the  remaining  subjects  respond  as  lis  - 
teners. 


Kgure  2.  INDIVIDUAL  VOCRES  STATION  (DESK) 


COMMUNICATION  MATERIALS 

Communication  materials  consist  of  the  standardized  Modified  Rhyme  Test*  for  most 
activities  with  VOCRES.  Other  test  materials  such  as  the  Diagnostic  Rhyme  Test  are  used 
from  time  to  time  for  special  purpose  applications. 

COMMUNICATION  LINK  CAPABILITIES 

The  communications  assemblage  diagram  (Fig.  3)  demonstrates  the  high  flexibility  of 
VOCRES  that  allows  a  variety  of  different  communication  links  to  be  examined  either 

*  Standardised  lists  of  50  monosyllabic  words;  each  list  developed  to  be  essentially  equivalent  in  intellieibilitY  to  the 
other  lists. 
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Figure  3.  BLOCK  DIAGRAM  VOCRES  COMMUNICATION  LINK 


Individually  or  in  combination  with  one  another.  The  range  of  communication  links  can  be 
varied  from  a  simple  face-to-face  communications  situation  (i.e.,  direct  talke1-  to  listener) 
to  a  complex  configuration  using  encoders,  encrypters,  and  the  like  by  varying  appropriate 
subunit  controls.  Any  of  the  alternate  pathways  shown  in  Figure  3  can  be  used  to 
complete  the  talker  to  listener  link.  The  direct  talker  to  listener  path  theoretically  provides 
a  data  baseline  free  from  environmental  and  component  effects.  The  influences  of  the 
.various  elements  of  the  communication  system  operation  relative  to  the  baseline  can  be 
quantified,  analyzed  and  evaluated  by  measuring  performance  while  varying  single 
components  and  clusters  of  components  of  the  VOCRES, 

CENTRAL  PROCESSOR-DISPLAY-RESPONSE  SYSTEM 

The  control  console  of  the  system  includes  a  typewriter  type  keyboard  and  a  cathode  ray 
tube  (CRT)  display.  Through  this  console  the  test  administrator  enters  the  required 
experimental  information.  The  central  processing  unit  then  displays  the  required 
instrument  on  the  CRT  (Fig.  4).  After  all  experimental  instrument  settings  are  completed 
and  stabilized,  the  administrator  tells  the  central  processing  unit  to  administer  the 
selected  test  and  collect  data  from  each  of  ten  individual  communications  desks.  The 
system  is  capable  of  making  any  one  of  the  10  stations  the  talker  position  and  also  can 
facilitate  multiple  talkers.  For  example,  during  a  test  one  subject  will  be  designated  a 
talker  for  a  list  of  50  words  and  the  othor  nine  subjects  will  bo  designated  as  listeners. 
On  the  CRT  the  system  displays  each  of  the  listeners'  responses  to  each  item  spoken  by 
the  talker.  The  CRT  display  also  indicates  whether  or  not  the  response  is  correct  (Fig.  5). 
The  central  processor-display-response  system  is  diagrammed  in  Figure  6.  The  central 
processor  is  the  Hewlett-Packard  9845T  System.  This  system  has  dual  16-bit  processors, 


wherein  one  handles  internal  functions,  while  the  second  handles  I/O  functions.  Also 
included  in  the  basic  system  are  a  CRT  with  graphics,  a  thermal  line  printer  (8- 1/2" wide) 
with  graphics  capability  and  two  cartridge  tape  drives,  each  with  217K  byte*  capacity.  A 
20  Mega  byte  disk  drive  with  two  platters,  one  fixed  and  one  removable,  adds  additional 
data  storage  capability.  Several  interfaces  are  also  included  in  the  system.  An  RS-232 
interface  is  used  for  sending  and  receiving  data  from  the  individual  communication 
stations,  while  an  IEEE  488-1975  General  Purpose  Interface  Bus  is  used  for  control  and 
data  collection  from  various  electronic  instruments.  These  include  a  digital  spectrum 
analyzer,  a  frequency  synthesizer,  a  digital  voltmeter,  an  RF  power  meter,  and  a  4-co!cr 
flat  bed  X-Y  plotter.  A  second  RS-232  interface  receives  data  from  a  digital  oscilloscope  or 
an  audio  tape  deck. 

Each  individual  communication  desk  has  its  own  RS-232  compatible  interface  shown  in 
Figure  4  &  5  which  decodes  commands  by  the  central  processor  for  the  display  system 
and  also  returns  the  subjects'  responses  to  the  central  processor  for  storage  and  analysis. 
Each  desk  station  interface  has  two  addresses  to  which  it  will  respond.  One  address  is 
common  to  all  desks,  therefore  by  using  one  address  and  message,  all  desk  displays  can 
be  activated  or  loaded  simultaneously.  The  second  address  is  specific  to  only  one  desk 
and  by  using  this  address,  ten  different  messages  can  be  loaded  into  each  of  ten  different 
displays.  The  interface  for  each  desk  operates  at  9600  bits  per  second  allowing  seemingly 
simultaneous  operation  at  each  of  the  ten  stations. 

Figure  7  shows  one  of  the  64  character  alphanumeric  gas  discharge,  type  displays.  Each 
character  is  5.73  mm  (.023  in)  x  8.27  mm  (0.33  in)  and  is  generated  by  a  5x7  dot  matrix 
with  a  separate  underline  capability.  The  display  is  very  bright  having  a  level  of  30  ft-L. 
The  contrast  of  the  neon-orange  characters  is  enhanced  by  the  use  of  a  circularly 
polarized  filter. 

The  subjects  can  respond  by  using  one  of  two  different  response  systems.  The  first 
system  consists  of  six  pushbuttons,  three  on  either  side  of  the  displays  each  with  a  red 
LED  mounted  in  the  bezel.  Pressing  one  button  causes  the  adjacent  LED  to  light  indicating 
a  response  has  been  made.  Pushing  a  second  button  will  allow  the  volunteer  to  change 
his  decision,  illuminating  the  second  light  instead  of  the  first.  The  second  response 
system  consists  of  two  4x4  calculator  type  keypads.  Only  one  of  the  32  buttons  can  ba 
chosen  at  one  time.  Operation  is  similar  to  the  six  LED  pushbuttons  except  that  pressing 
one  of  the  keys  causes  from  one  to  five  of  the  six  LEDs  to  light  forming  a  specific  pattern 
■for  that  key.  These  LEDs  provide  feedback  to  the  subject  indicating  the  chosen  response. 

DATA  TREATMENT 

Computer  software  was  developed  to  standardize  test  procedures  and  to  facilitate  the 
administration  of  the  Modified  Rhyme  Test  or  any  other  standardized  intelligibility  test 
over  a  Irrge  number  of  individual  trials.  The  software  also  includes  the  experimental 
design,  each  test  parameter  is  displayed  on  the  CRT  before  the  trial  and  appropriate 
equipment  settings  are  made  by  both  the  test  administrator  and  central  processing  unit. 
The  individual  units  of  the  Modified  Rhyme  Test  or  any  r  ther  test  materials  are  stored  on 
the  system's  20  Mega  Byte  hard  disk.  Following  each  trial,  data  for  each  subject,  all  test 
narameters  and  the  time  of  the  trial  are  stored  on  the  system's  disk.  Fail-safe  backup  is 
accomplished  by  printing  the  same  data  on  the  system's  thermal  line  printer.  The  data 


•  Note:  1  byw  =  8  bits 
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Figure  7.  INDIVIDUAL  DESK  DISPLAY  UNIT 


may  be  analysed  at  any  time,  using  a  variety  of  standard  statistical  measures  and  plotting 
.techniques.  This  method  of  data  storage  and  analysis  can  give  preliminary  results  in  ieal 
time. 


HIGH  INTENSITY  SOUND  SYSTEM 

The  high  intensity  sound  system  is  shown  in  Figures  8  and  9.  The  system  is  capable  of 
operation  in  one  of  two  power  modes,  a  high  power  mode  whore  14,000  watts  are 
available  and  a  low  power  mode  where  1,200  watts  are  available.  The  power  amplifiers 
drive  eight  banks  of  loudspeakers  containing  a  total  of  96  Altec  15"  .ow-frequency 
speakers,  eight  Altec  horn  loaded  compression  drivers,  and  384  Stromberg  Carlson  high 
frequency  speakers.  The  noise  generator  and  the  spoctrum  shaper  allow  almost  anv 
desired  noise  environment  (spectrum)  within  the  human  audio-frequency  range  to  be 
generated  inside  the  test  chamber.  This  permits  the  accurate  reproduction  of  ambient  and 
environmental  noise  conditions  of  specific  operational  situations  within  the  laboratory, 
which  is  a  vital  aspect  of  the  validity  of  the  communication  testing. 

The  room  in  which  the  loudspeaker  banks  are  located  is  a  specially  designed  and 
constructed  acoustic  reverberation  chamber.  The  room  is  designed  for  maximum 
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reverberation  time  and  approximately  8000  ft3  in  volume.  The  'irregular  wall  surfaces  are 
designed  to  disrupt  the  formation  of  standing  waves  and  maximize  the  uniformity  of  the 
level  of  a  noise  distributed  throughout  the  room. 


AIC-25  INTERCOMMUNICATION  SYSTEM 

The  aircraft  intercommunication  system  shown  in  Figure  10  is  a  standard  AIC-25 
intercommunication  system.  The  test  administrator  and  each  desk  has  ar.  in¬ 
dividual  AIC-25  aircraft  intercommunication  unit  .  A  switching  circuit  located  on  the 
control  console  allows  the  talker's  intercom  to  be  disconnected  from  the  rest  of  the 
system  and  taken  directly  to  the  audio  input  of  any  transmitter.  The  audio  output  of  the 
receiver  is  then  routed  to  the  other  nine  listeners.  The  terminal  equipment  available  for 
the  intercom  system  includes  standard  H-157A  headsets,  H-133  headsets,  MBU-5/P 
oxygen  masks,  and  KGU-26/P  flight  helmets.  A  sample  of  each  of  these  is  pictured  in 
Figure  11. 


AIR  RESPIRATION  SYSTEM 

The  air  breathing  system  depicted  in  Figure  12  uses  the  standard  Air  Force  A-19  diluter 
demand  regulator  as  the  primary  item  in  the  system.  Each  station  has  its  own  A-19 
regulator  which  is  supplied  through,  feeder  lines  by  a  semiautomatic  regulator  manifold. 
The  manifold  connects  six  standard  size  breathing  air  bottles  to  the  system  through  two 
regulators.  Each  regulator  controls  three  bottles.  When  the  supply  of  the  first  three  bottles 
is  exhausted  the  system  automatically  switches  to  the  second  set  of  three  bottles.  The 
normal  operating  pressure  in  the  system  is  150  psig. 

Each  of  the  above  systems  is  integrated  into  each  of  ten  individual  subject  stations.  The 
final  product  is  shown  in  Figures  13  and  14.  The  desk  was  designed  for  minimum  size  to 
minimize  acoustical  reflections  from  the  surface  and  yet  be  functional.  Each  station  is 
independent. 

In  the  past,  interim  versions  of  the  VOCRES  system  were  used  to  evaluate  communication 
properties  of  lightweight  helmets,  chemical  defense  ensembles,  new  oxygen  masks,  and 
innovative  radio  systems.  Current  studies  involve  the  investigation  of  effects  of  jamming 
on  communication  in  a  quantitative  manner  relative  to  the  J/S,  S/N,  radio  type  and 
jammer  type,  evaluation  of  new  chemical  defense  ensembles,  and  development  of  new 
communication  microphones.  Future  studies  will  include  modeling  of  human  response  to 
jamming  and  enhancement  of  terminal  communication  equipment. 


Figure- 1 3.  INDIVIDUAL  VOCRES  STATION  (DESK) 


SUMMARY 


This  paper  has  described  the  VOCRES  system,  its  capabilities  and  uses.  In  summary, 
VOCRES  is  a  semiautomatic  laboratory  voice  communication  test  system  that  uses  human 
subjects  in  a  realistic  communication  environment  to  conduct  research,  test  and 
evaluation  of  Air  Force  communications  systems  and  their  effectiveness. 
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TEST  PLAN 


AFIT  LS  THESIS  *89-83: 

ACCURACY  AND  SPEED  OF  RESPONSE  TO  DIFFERENT  VOICE  TYPES 
IN  A  COCKPIT  WARNING  SYSTEM 

1.  Experimental  Test  Planning  Documentation 

a.  Test  Objective  and  Purpose 

The  purpose  o-f  this  study  is  to  investigate  the 
effects  of  different  voice  types  (male,  female,  and  machine) 
on  the  speed  and  accuracy  of  'response  to  a  voice  warning 
system.  The  results  of  this  study  will  be  used  in  the  design, 
development,  and  selection  of  auditory  advisory  annunciator 
systems  for  military  aircraft. 

b .  Experi mental  Design 


The  variables  to  be  investigated  are  3  voice  types 
(male,  female,  machine) .  The  moderating  variables  are  warning 
format,  background  noise,  and  signal-to-noise  ratio.  Warning 
format  will  take  three  forms:  tone  precursor,  voice  precursor 
(the  word  "Warning"),  and  repeated  warning,  in  which  the 
warning  acts  as  its  own  precursor.  Background  noise  will  be 
105  db  and  115  db,  which  represent  two  high-noise  situations 
in  an  F-16A  cockpit.  Signal-to-noise  ratios  will  be  0,  5,  and 
10  db  above  headphone  conversation  noise,  as  delivered  to  the 
headphones  over  the  intercomm  system.  All  subjects  will  be 
tested  under  every  experimental  condition  (54  combinations). 

c.  Experimental  Procedures 

Procedures  are  detailed  in  AFIT  LS  Thesis  #89-83 
Speed  and  Accuracy  of  Response  to  Different  Voice  Types  in  a 
Cockpit  Warning  System.  Chapter  III  (Methodology). 

There  will  be  ten  subjects,  recruited  by  the  AMRL. 
The  only  restrictions  on  the  subjects  are  that  they  possess 
good  hearing  as  measured  by  a  standard  audiometric  screening 
test  and  that  they  can  participate  in  all  54  tests.  During 
the  test,  subjects  will  be  required  to  perform  two  tasks:  a 
primary  "tracking"  task,  and  a  secondary  "emergency  response" 
task. 

The  primary  task  will  consist  of  pushing  the  button 
closest  to  a  marker  on  the  LED  display.  The  location  of  the 
marker  continually  changes  in  a  random  manner. 

The  secondary  task  will  be  to  correctly  respond  to 
emergency  warnings,  which  will  be  interjected  peri odi cal  1 y. 

The  warnings  will  consist  of  instructions  specifying  which  one 
of  32  buttons,  located  on  a  pad  below  the  LED  display,  must  be 
pushed.  Each  of  the  32  buttons  will  be  chosen  at  least  once 
during  each  of  the  54  tests. 

During  the  course  of  each  test  session,  data  will  be 
collected  by  a  Hewlett  Packard  9845T  desktop  computer  system. 


A  a-' 
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The  data  Will  subsequently  be  transferred  to  the  AFIT/ASD  CDC 
computer  -for  analysis-  Noise  exposure  conditions  at  the  ear 
are  within  the  limits  specified  by  AFR  161-35,  Hazardous 
Noise  Exposure?  and  are  nonhazardous.  Procedures  used  are 
in  accordance  with  AFR  .169-3,  Use  o-f  Human  Subjects  in 


RBT&E. 


d.  Test  System  Requirements 


The  standard  Voice  Communication  Research  and 
Evaluation  System  (VOCRES)  facilities  will  be  augmented  with  a 
digitized  voice  capability,  generated  by  the  Texas  Instruments 
<TI)  5220  speech  synthesis  chip-  Reference  Voice 
C-ommun  icat ion  Research  and  Evaluation  System 
(AFAMRL— TR-80-25) ,  May  19S0. 

e-  Data  Processing  Techniques 

Data  will  be  processed  on  the  AFIT/ASD  CDC 
computer,  using  the  Statistical  Package  for  the  Social 
Sciences  (SPSS).  The  SPSS  program  will  calculate  mean 
response  times  and  accuracy  for  each  condition,  and  indicate 
any  appropriate  correlations  in  the  data  set. 

f.  Documentation  Reauirements 


This  experiment  is  being  documented  under  AFIT  LS 
Thesis  #39-83.  This  experiment  is  being  performed  in  support 
of  the  Master's  Degree  program  in  Systems  Management. 

g.  Mr.  Timothy  Anderson  and  Mr.  Richard  McKinley  will  be 
responsible  for  calibrating  the  test  equipment,  generating 
noise  and  test  signals.  Major  Freedman  and  Captain  Rumbaugh 
will  be  responsible  for  analysis  and  i nterpretati on  of  the 
data. 


h .  Responsibilities  of  Technical  Service 
Organization 

BBE  will  be  responsible  for  maintaining  the  high 
intensity  sound  systems. 

i .  Responi bi 1 i ti es  of  WFAFB  Support  Organizations 

None 

j .  Human  Use  Protocol 

#7’S-13  "Nonhazardous  Human  Exposure  to  Acoustic 

Energy" 
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Instrumentation  Calibration  Procedures 


Instrumentation  shall  be  calibrated  in  accordance 
with  BBE  SOP  #15. 

1 .  Instrumentation  Calibration  Records 


Calibration  will  be  recorded  at  the  beginning  and 
end  of  each  test.  Records  of  calibration  will  be  kept  with 
the  test  data. 

m.  Facility  Operational  Procedures 

Procedures  shall  be  conducted  in  accordance  with  BBE 

SOP  #14. 


n.  Facility  Operational  Checklists 

VOCRES  -facility  operational  checklists  will  be 
provided  by  MR.  Timothy  Anderson  and  Mr.  Richard  Mckinley. 

o.  Description  o-f  Data  Collection  Systems 


Speed  and  accuracy  o-f  response  data  will  be 
collected  by  means  of  Hewlett  Packard  9845T  desktop  computer. 
A  hard  copy,  tape  copy,  and  disk  copy  will  be  retained. 

p.  Test  Schedule 


Testing  will  commence  on  or  about  June  1,  19S3. 

q .  Safety  and  Emergency  Procedures 

AMRL/BB  Safety  Officer  has>  and  will  continue  to 
conduct  monthly  safety  inspections  of  experimental  area. 

2.  N/A 

3.  Major  Freedman  and  Captain  Rumbaugh  will  be 
co-investigators.  Experiments  will  be  conducted  under  the 
guidance  of  Mr.  Anderson  and  Mr.  McKinley. 

4.  Mr.  Anderson  and  Mr.  McKinley  will  be  the  on-site  operating 
officials. 
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EXPERIMENTAL  DESIGN  SEQUENCE 


The  combination  of  parameters  and  the  original  order 
were  determined  using  random  methods  of  selection.  The 
random  order  thus  arrived  at  is  reflected  in  the  "run 
parameter"  code  number.  The  combinations  were  then 
adjusted  only  when  necessary  to  remain  within  USAF 
standards  of  noise  exposure,  as  established  by  AFR  161-35 
Hazardous  Noise  Exposure ,  and  AFR  169-3,  Use  of  Human 
Subjects  in  RDT&E.  The  final  sequence  is  indicated  by 
the  "run  sequence  number". 


RUN 

RUN 

VOICE 

BACKGROUND 

S/N 

PRECURSOR 

SEQUENCE 

NUMBER 

PARAMETER 

TYPE 

NOISE  <db) 

RATIO  (db) 

TYPE 

1 

5 

Machine 

115 

0 

Voice 

2 

6 

Male 

115 

10 

Tone 

3 

13 

Male 

105 

10 

Tone 

4 

10 

Female 

115 

10 

Repeated 

5 

9 

Male 

115 

5 

Voice 

6 

11 

Machine 

105 

0 

Tone 

7 

14 

Female 

105 

10 

Voice 

8 

1 

Male 

115 

0 

Repeated 

9 

2 

Machine 

115 

5 

Tone 

10 

3 

Machine 

105 

0 

Voice  ■ 

11 

15 

Female 

105 

10 

Repeated 

12 

4 

Machine 

115 

0 

Tone 

13 

7 

Mai  e 

115 

5 

Tone 

14 

16 

Female 

105 

5 

Repeated 

15 

17 

Female 

105 

0 

Tone 

16 

8 

Male 

115 

0 

Voice 

17 

12 

Femal e 

115 

5 

Tone 

18 

18 

Male 

105 

0 

Repeated 

19 

19 

Machine 

105 

0 

Repeated 

20 

20 

Female 

115 

0 

Voice 

21 

21 

Machine 

105 

10 

Repeated 

22 

22 

Female 

105 

0 

Voice 

23 

24 

Female 

115 

5 

Voice 

24 

23 

Male 

105 

5 

Tone 

25 

25 

Male 

115 

10 

Repeated 

26 

26 

Machine 

105 

10 

Tone 

27 

27 

Female 

115 

0 

Repeated 

28 

28 

Machine 

115 

10 

Tone 

29 

29 

Machine 

115 

5 

Repeated 

30 

32 

Male 

105 

10 

Repeated 

31 

30 

Female 

115 

0 

Tone 

32 

31 

Mai  e 

115 

5 

Repeated 

33 

35 

Male 

105 

10 

Voice 

34 

37 

Female 

105 

10 

Tone 

35 

33 

Machine 

115 

5 

Voice 

RUN 

RUN 

VOICE 

BACKGROUND 

S/N 

PRECURSOR 

SEQUENCE 

NUMBER 

PARAMETER 

TYPE 

NOISE  fdb) 

RATIO  (db) 

TYPE 

36 

34 

Machine 

115 

10 

Repeated 

37 

39 

Mai  e 

105 

0 

Tone 

38 

41 

Female 

105 

5 

Voice 

39 

36 

Male 

115 

0 

Tone 

40 

38 

Femal e 

115 

10 

Voice 

41 

42 

Femal e 

105 

5 

Tone 

42 

43 

Machine 

105 

5 

Tone 

43 

44 

Male 

105 

5 

Voi  ce 

44 

45 

Machine 

115 

10 

Voice 

45 

40 

Female 

115 

5 

Repeated 

46 

48 

Machine 

105 

10 

Voice 

47 

46 

Female 

115 

10 

Tone 

48 

47 

Machine 

115 

0 

Repeated 

49 

49 

Machine 

105 

5 

Repeated 

50 

50 

Machine 

105 

5 

Voice 

51 

51 

Male 

115 

10 

Voice 

52 

52 

Male 

105 

5 

Repeated 

53 

5o 

Male 

105 

0 

Voice 

54 

54 

Female 

105 

0 

Repeated 
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The  results  o-f  this  experiment  will  be  used  to  help 
develop  cockpit  voice  warning  systems  -for  tactical  aircraft 
(-fighters).  The  experiment  consists  of  54  runs,  each 
lasting  about  35  minutes.  Testing  is  expected  to  take 
about  2  1/2  weeks. 

The  experiment  uses  two  tasks:  a  primary  task  associated 
with  the  display,  and  a  secondary  task  associated  with  the 
number  pad.  The  primary  task  consists  of  pressing  the 
display  button  nearest  a  light  appearing  on  the  display 
(see  figure  1).  It  may  help  to  think  of  the  display  being 
divided  into  eight  sections,  left  and  right  halves  of  the 
display  being  divided  into  four  rows  each  (top,  upper 
middle,  lower  middle,  and  bottom).  For  an  example,  when  a 
light  appears  anywhere  in  the  bottom  left  section,  press 
button  4;  if  it  appears  in  the  upper  middle  right  section, 
press  button  6,  and  so  on. 


LEFT  RIGHT 


Figure  1.  Top  Panel 

The  light  may  appear  in  different  areas  of  each  section, 
and  the  lighted  section  will  be  changing  rapidly.  It  will 
be  necessary  to  work  quickly  and  it  may  help  to  use  two 
hands. 
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At  random  times  during  each  run,  a  voice  over  the 
headphones  will  announce  the  side,  color,  and  number  of  a 
button  on  the  number  pad  (see  figure  2) . 


LEFT  RIGHT 
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Figure  2.  Number  Pad 


Quickly  press  the  designated  button  as  rapidly  as  possible. 
While  responding  to  the  voice  alert  as  rapidly  as  possible 
do  not  neglect  the  primary  task.  Think  of  a  pilot  who  must 
take  corrective  action  during  a  flight  emergency  (the 
secondary  task) ,  yet  must  continue  to  fly  the  aircraft  (the 
primary  task). 

It  is  expected  that  sl; — •*  people  will  do  better  than  others 
and  that  techniques  will  differ.  During  the  practice  runs, 
try  several  different  methods  or  strategies  for 
accomplishing  both  tasks:  for  example  use  one  hand,  use 
two  hands,  always  respond  to  the  voice  alert  with  the  same 
hand,  respond  to  the  voice  alert  with  different  hands,  etc. 
During  the  practice  period  try  to  determine  which  method  or 
strategy  works  best  for  you.  After  selecting  the  method  or 
strategy,  practice  trying  to  gain  experience  with  it  during 
the  remainder  of  the  practice  period.  Once  the  practice 
period  is  over  do  not  change  methods  or  strategies.  After 
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selecting  the  technique  which  is  most  corat ortabl e ,  use  that 
same  technique  -for  all  of  the  runs. 
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APPENDIX  C± 


CONSRAW 


PROGRAM  CONSRAW  (INPUT/,  OUTPUT,  SRAWTOT ,  C 1 R I L , SUBF I L , GPF I L ,  SCAT) ; 


(*  READS  "SRAWTOT"  ■  *) 

(*  MANIPULATES  DATA,  AND  OUTPUTS  TO  MANY  FILES  *) 

(*==============r=============r ======================================*) 

(«  NOTE .  *> 

(*  TO  CUSTOMIZE  INPUT,  RESET  THE  CONSTANT  "R"  (NUMER  OF  RUNS)  *> 

(*  IN  THE  "SETUP"  PROCEDURE.  *) 

(*  TO  CUSTOMIZE  OUTPUT,  MERELY  DELETE  THE  CALL  FOR  A  PARTICULAR  *) 

(*  OUTPUT  PROCEDURE  IN  THE  LAST  SECTION  OF  THE  PROGRAM.  *) 

(*===================================================================*) 


(*  FOR  FORMAT  OF  INPUT  ?<  OUTPUT  FILES,  SEE  "FILE  CONVENTIONS"  FILE  *) 
{********************************************************* a****#*#**#*) 


UPSEUDO  CODE:  *) 
(*  Set  up  files  and  variables  #) 
(*  Read  input  CSRAWT0T3  *) 
(*  Do  data  manipulations  *) 
(*  **For  each  run',  do:  *) 
(#  *For  each  subject, do:  *) 
(*  Calculate  primary  task  Coveral 1 -basel inel  figures  *) 
(*  Calculate  accuracies  [percentages!  *) 
<*  {primary,  overall,  baseline,  reduction,  percent-BLl  *) 
(*  Compare  warning-responses  to  correct  responses  *) 
(*  Assign  "Flag  Codes"  Cl, 0,8!  to  responses  *) 
(*  Create  time-response  array  [matrix!  *) 
(#  Calculate  accuracy  (percent)  and  average  time  *) 
(*  *For  Group,  calculate:  *) 
(*  Primary  task  baseline  accuracy,  *) 
(*  Primary  accuracy,  *) 
(*  Primary  reduction,  *) 
(*  Primary  percentage  of  baseline,  •  *) 
{#  Warning  task  accuracy,  *) 
(*  Warning  mean  reasponse-time  [for  correct  responses  only!  *) 
<*  Write  output  CCiFIL,SUBFIL,GPFIL,SDAT!  *) 


(#**#***#*****#***##***##*###**#*****####*###**###******###******#***#) 

CONST 

BLANK  =  '  '? 

WARNSUM  =  32; 

S  =  10;  (*  S  =  NUMBER  OF  SUBJECTS  FOR  EACH  RUN  *) 

R  =  54;  (*  R  =  NUMBER  OF  RUNS  IN  SRAWTOT  *) 
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TYPE 


INTARRAY  =  ARRAY  C1..R,  1..S3  GF  INTEGER; 
REALARRAY  =  ARRAY  C1..R,  1..S3  OF  REAL; 
RUNARRAY  =  ARRAY  E1..R3  OF  REAL; 

REACTREC  =  RECORD 

RESPONSE  :  INTEGER; 

RESPTIME  :  REAL; 

END;  (*R£ACTREC*) 

LINEREC  =  RECORD 

LINENUM  :  INTEGER; 

ANSWER  :  ARRAY  El.. 83  OF  REACTREC; 
END;  ULINEREC*) 

SU8JECTREC  =  RECORD 

FIRSLINENUM  :  INTEGER; 

SUBNUM  :  INTEGER; 

TRAKTOT  ;  INTEGER; 

TRAKCORR  :  INTEGER; 

BLTOT  :  INTEGER; 

BLCORR  :  INTEGER; 

REACTION  ;  ARRAY  Cl.. 43  OF  LINEREC; 

END;  (*SUBJECTREC*) 

TIMEREC  =  RECORD 
YEAR  !  INTEGER; 

MONTH  s  INTEGER; 

DATE  ;  INTEGER; 

HOUR  !  INTEGER; 

MINUTE  s  INTEGER; 

SECOND  ;  INTEGER; 

END;  UTIMEREC  »> 

RUNREC  =  RECORD 

RECLINENUM  :  INTEGER; 

RUNSEQNUM  :  INTEGER; 

TIMEDAT  :  TIMEREC; 

RUNPAR  ;  INTEGER; 

VTYPE  i  CHAR; 

BKGRD  :  INTEGER; 

SIGNOISE  ;  INTEGER; 

PRECURS  ;  CHAR; 

STIMULUS  :  ARRAY  Cl.. 323  OF  INTEGER; 
SUBDAT  :  ARRAY  Cl.. S3  OF  SUBJECTREC; 
END;  (*  RUNREC  *) 
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VAR 


SRAWTOT ,  C1FIL  ;  TEXT; 

SUBFIL,  GPFIL  :  TEXT; 

SDAT  s  TEXT; 

STIMRESP  :  ARRAY  C1..R,  1..S,  1..323  OF  INTEGER; 

WARNRESP  :  ARRAY  E1..R,  1..S,  1..323  OF  INTEGER; 

WARNTIME  :  ARRAY  C1..R,  1..S,  1..323  OF  REAL; 

RUNDAT  ;  ARRAY  C1..R3  OF  RUNREC; 

G,  H,  I,  J,  K,  L,  M,  N  :  INTEGER; 

STIMNUM  ;  INTEGER; 

PRTOT ,  PRCORR  :  INTARRAY; 

GPPRTOT ,  GPPRCORR,  GPBLTGT ,  GPBLCORR  s  ARRAY  C1..R3  OF  INTEGER; 
WARNCORR,  NORESPSUM  :  INTARRAY; 

GPWARNSUM ,  GPWARNCORR ,  GPNCRESPSUH  :  ARRAY  E1..R3  OF  INTEGER; 

WARNACC,  BLTRACC,  PRTRACC,  PRACCLOSS,  PRPERCBL  :  REALARRAY; 
MNRESPTIME ,  TIMESUM  s  REALARRAY; 

GPBLTRACC,  GPPRTRACC ,  GPPRACCLOSS,  GPPRPERCBL  :  RUNARRAY; 
GPMNWARNCORR,  GPWARNACC ,  GPMNRESPTIilE  :  RUNARRAY; 

GPTIMESUM,  GPMNNORESPSUM  :  RUNARRAY; 

SPACER  s  CHAR; 


<*********#******#***#******************************#*****************) 
PROCEDURE  SETUP; 


(•PSEUDO  CODE:  *) 
(*  Set  up  files  *) 
(*  Reset  input  files  #) 
(*  Rewrite  output  files  *) 
(*  Initialize  variables  for  summing  *) 
(* . . . . . . . . *) 


BEGIN 

RESET  (SRAWTOT); 

REWRITE  (ClFIL)j 
REWRITE  (SUBFIL); 

REWRITE  (GPFIL); 

REWRITE  (SDAT); 

FOR  (1  :*  1  TO  R  DO  BEGIN 
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(*  PRIMARY  TASK  SUMS  *) 


GPPRT0TCM3  :=  0; 

GPPRCQRRCM3  :=  0; 

GP8LT0TCM3  :=  0; 

GPBLC0RRCM3  :=  0; 

(*  SECONDARY  TASK  SUMS  *) 

FOR  I  :=  1  TO  S  DO  BEGIN 
TIMESUMCM, I]  :=  0.0; 

WARNCORRCM ,  1 1  ;=  0; 

NORESPSUMCM, 13  :=  0; 

END;  (*FOR  I... BEGIN  *) 

GPWARNSUMCM3  :=  0; 

GPWARNC0RRCM3  :=  0; 

GPN0RESPSUMCM3  :=  0; 

GPTIMESUMCM3  ;=  0.0; 

END;  (*  FOR  M.. .BEGIN  *> 

END;  (*  SETUP  *) 

(*******#********************* ***r a***************************** ##****) 
PROCEDURE  READITIN; 


(*PSEUDO  CODE  *) 
(*  For  each  run,  do;  *) 
(*  Read  record  00  *) 
<*  Read  32  "warning"  stimuli  into  (STIMULUS)  *) 
(*  For  each  subject  do  *) 
(*  Read  five-record  set  [must  read  f ield-by-f ield3  *) 
(*  Format  into  proper  array  information  #) 
(* . *) 


BEGIN 

FOR  M  ;=  1  TO  R  DO  BEGIN 
WITH  RUNDATCM3  DO  BEGIN 
READ  (SRAWTOT ,  RECLINENUM,  RUNSEQNUM) ; 

WITH  TIMEDAT  DO 

READ  (SRAWTOT,  YEAR,  MONTH,  DATE,  HOUR,  MINUTE, 

SECOND); 

READ  (SRAWTOT,  RUNPAR ,  .SPACER ,  VTYPE  ,  BKGRD ,  SIGNQISE, 

SPACER,  PRECURS); 

READLN  (SRAWTOT); 

(*  CHECKS  THAT  RUNS  ARE  ENTERED  IN  ORDER  BY  COMPARING  RUNSEQNUM  *) 
(*  WITH  PREVIOUS  RUNSEQNUM,  AND  PRINTS  WARNING  ON  SCREEN  *) 

IF  M  >  1  THEN  BEGIN 
G  ;=  M  -I; 

IF  RUNSEQNUM  <>  RUNDATCG3. RUNSEQNUM  +  1  THEN 

WRITELN  ('NOTE;  RUN  SEQUENCE’,  RUNSEQNUM,'  LISTED  OUT  OF  ORDER’); 
END;  (*  IF  M  *) 
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READ  (SRAWTOT,  RECLINENUM); 

FOR  I  :=  i  TO  16  DO 

READ  (SRAWTOT.  STIMULUS! 1 3 ) ; 

READLN  (SRAWTOT); 

READ  (SRAWTOT,  RECLINENUM); 

FOR  I  :=  17  TO  32  DO 

READ  (SRAWTOT,  ST IMULUSC I ] ) ; 

READLN  (SRAWTOT); 

(*  READ  WARNING  RESPONSE  DATA  FOR  EACH  SUBJECT  *) 

FOR  J  :=  1  TO  S  DO  BEGIN 

N  :=  1;  <*  RESETS  STIMULUS  NUMBER  *> 

WITH  SUBDATE J]  DO  BEGIN 

READ  (SRAWTOT,  FIRSLINENUM,  SUBNUM,  TRAKTOT,  TRAKCORR, 

BLTOT ,  BLCORR) ; 

READLN  (SRAWTOT); 

FOR  K  :=  1  TO  4  DO  BEGIN 

WITH  REACTIONCK3  DO  BEGIN 
READ  (SRAWTOT,  LINENUM); 

FOR  L  :=  1  TO  8  DO  BEGIN 
WITH  ANSWERCL3  DO  BEGIN 

READ  (SRAWTOT,  RESPONSE,  RESPTIME): 

<*  ASSIGNS  DATA  TO  NEW  ARRAYS  *) 

STIMRESPtM, J ,N]  :=  RESPONSE; 

WARNTIMECM, J ,N]  ;=  RESPTIME; 

IF  RESPONSE  =  ST IMULUSCN3  THEN 
WARNRESPEM , J , N3  :=  1 
ELSE  IF  RESPONSE  =  33  THEN 
WARNRESPCM, J ,N3  ; =  8 
ELSE  IF  RESPONSE  =  99  THEN 
WARNRESPEM, J ,N3  ;=  8 
ELSE  WARNRESPtM, J,N3  :=  0; 

N  :=  N  +  l;  (*  UPDATES  WARNING  NUMBER  *) 

END;  (*  WITH  ANSWER. . ; BEGIN  *> 

END;  (*  FOR  L. . .BEGiN  *) 

READLN  (SRAWTOT); 

END;  (*  WITH  REACTON. .. BEGIN  *) 

END;  <*  FOR  K... BEGIN  *) 

END;  (*  WITH  SU8DAT, .. BEGIN  *) 

END;  (*  FOR  J.. .BEGIN  *) 

END;  (*  WITH  RUNDAT. .. BEGIN  *) 

END;  (*  FOR  M... BEGIN  *) 

END?  OREADITIN*) 

(**##*****#**#***#***********#******##*#***#***#*****#****************) 
PROCEDURE  CALCULATE; 
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UPSEUDO  CODE 


(* 

(* 

(* 

(* 

(* 

(* 

(* 

(* 

(* 

<* 

(* 

(* 

(* 

(* 

(* 

(* 

(* 

(* 

(* 

<* 

(* 

(* 

(* 

(* 

<* 

(* 


For  each  run,  do: 

For  each  subject,  calculate: 

Primary  (tracking)  task: 

Primary  (Non-BASELINE)  stimuli  ?<  responses 
Baseline  accuracy  (percentage) 

Primary  accuracy 

Accuracy  loss  [baseline  minus  primary] 

Primary  percent  of  peak  performance  ( i . e. , BASELINE) 
[Primary  divided  by  Baseline] 

Secondary  (warning)  task: 

Number  of  correct  responses 

Number  of  "No-response  within  allotted  time" 

Percent  correct  responses 

Mean  response  time  (correct  responses  only) 

For  the  Group  (for  each  run),  calculate 
Primary  task: 

Mean  primary  accuracy 
Mean  baseline  accuracy 
Mean  accuracy  loss 
Mean  percent  of  peak  performance 
Secondary  task: 

Mean  number  of  correct  reponses 

Mean  number  of  "no-responses  within  allotted  time" 
Mean  accuracy 
Mean  response  time 


*) 

*) 

*) 

*) 

*) 

*> 

*) 

*) 

*) 

*) 

*) 

*) 

*) 

*) 

*> 

*) 

*) 

*) 

*) 

*) 

*) 

*) 

*) 

*) 

*) 

*) 

*) 


BEGIN 

(*  FOR  EACH  RUN  *) 

FOR  M  :=  1  TO  R  DO  BEGIN 
WITH  RUNDATIM]  DO  BEGIN 

(*  FOR  EACH  SUBJECT:  *) 

FOR  J  :=  1  TO  S  DO  BEGIN 

(*  CALCULATES  PRIMARY  [NON-BASELINE]  PERFORMANCE  *) 
PRT0T[M,J]  :=  SUBDAT[J].TRAKTOT  -  SUBDATI J ] , BLTOT s 
PRCORRI M , J ]  :=  SUBDATI J ] . TRAKCORR  -  SUBDATI J  ] . BLCORR: 

(*  CALCULATES  ACCURACY  *) 

BLTRACCCM, J]  :=  SUBDATIJ ]. BLCORR/ SUBDAT [ J ]. BLTOT s 
PRTRACCIM ,  J  ]  :=  PRC0RR[M ,  J  ] /PRTQUM,  J  ]  | 

(*  ACCURACY  LOSS  *) 

PRACCLOSSIM, J]  :=  BLTRACCCM , J ]  -  PRTRACCIM , J ] ; 

(*  PERCENTAGE  OF  PEAK  [BASELINE]  PERFORMANCE  *) 
PRPERCBLIM, J]  :=  PRTR ACC C M , J ] / BL TR ACC C M , J ] : 
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(*  SECONDARY  [WARNING]  TASK  PERFORMANCE  *) 

(*  RESETS  SUMS  TO  ZERO  FOR  EACH  SUBJECT  *) 

WARNCORRCM.J]  :=  0; 

NORESPSUMCM , J ]  :=  b; 

TIMESUMCM , J ]  :=  0.0; 

FOR  N  i=  1  TO  32  DO  BEGIN 
CASE  WARNRESPCM , J , NJ  OF 
1  :  BEGIN 

WARNCORRCM , J ]  :=  WARNCORREM , J ]  +  1; 

‘  TIMESUMCM. J]  :=  TIMESUMCM, J]  +  WARNTIMECM, J ,N]; 

END;  (*  CASE  1  [CORRECT  RESPONSE]  *) 

0  :  ;  (*  NO  UPDATES  [INCORRECT  RESPONSE]  *) 

8  :  NORESPSUMCM, J]  :=  NORESPSUMCM, J ]  +  1;  (*  NO  RESPONSE  *> 
END;  (*  CASE  *) 

END;  (*  FOR  N... BEGIN  *) 

(*  COMPUTES  ACCURACY  &  MEAN  RESPONSE  TIME  FOR  CORRECT  ANSWERS  *> 
WARNACCCM , J ]  i=  WARNCORRCM, J ] / 3 2 : 

MNRESPTIMECM.J]  :=  TIMESUMCM, J ] /WARNCORRCM , J] j 

<*  NOTE:  GROUP  SUMS  WERE  RESET  TO  ZERO  IN  PROCEDURE  "SETUP"  *) 

(*  UPDATES  GROUP  SUMS  *) 

GPPRTOTCM]  :=  GPPRTOTCM]  +•  PRTOTCH, J3f 
GPPRCORRCM]  :=  GPPRCORRCM]  +  PRCORRCM , J ] ; 

GPBLTOTCM]  :=  GPBLTQTCM]  +  SUBDATC J ] . BLTOT; 

GPBLCORRCM]  :=  GPBLCORRCM]  +  SUBDATC J ]. BLCORR; 

GPWARNSUMCM]  :  =  GPWARNSUMCM]  +  32; 

GPWARNCORRCM]  :=  GPWARNCORRCM]  +  WARNCORRCM , J ] ; 

GPTIMESUMCM]  ;=  GPTIME3UMCM]  +  TIMESUMCM,]]; 

GPNORESPSUMCM]  :=  GPNORESPSUMCM]  +  NORESPSUMCM , J ] ; 

END;  <*  FOR  J. . .BEGIN  *) 

END;  (*  WITH  RUNDAT. .. BEGIN  *> 

(*  CALCULATES  GROUP  PERFORMANCE  *) 

<*  PRIMARY  [TRACKING]  TASK  *) 

GPPRTRACCCM]  :=  GPPRCORRCMJ/GPPRTOTCM]; 

GPBLTRACCCM]  :=  GPBLCORRCM] /GPBLTOTCM] : 

GPPRACCLOSSCM]  :=  GPBLTRACCCM]  -  GPPRTRACCCM 3 ; 

GPPRPERCBLCM]  :=  GPPRTRACCCM]/GPBLTRACCCM] ; 

(*  SECONDARY  [WARNING]  RESPONSE  *) 

GPMNWARNCORRCM]  ;  =  GPWARNCORRCM]/S: 

GPWARNACCCN]  :=  GPMNWARNC0RRCM]/32; 

GPMNRESPTIMECM]  :=  GPTIMESUMCM] /GPWARNCORRCM] ; 

GPMNNORESPSUMCM]  :=  GPNORESPSUMCMJ/S; 

END;  (*  FOR  M... BEGIN  #) 

END;  (*  CALCULATE  *) 


(ft************  ft***#**##*##*##******####  ft**#*##*  *##»*##*#**#* *********  ft) 


PROCEDURE  WRITITOUT; 


(ftPSEUDO  CODE  *) 
(*  For  each  run.  do:  *) 
(*  Write  output  to  "C1FII":  ft) 
(*  Writes  run-i denti f i ers  *) 
(*  For  each  subject  writes:  *) 
(*  Response  "flags"  Cl=correct,  O=incorrect,  8=no  response]  ♦  ) 
(»  Response  times  C  4  lines  @  10-10-10-2  3  ft) 
<♦  For  each  subject,  writes  summary  performance  information  ft) 
(*  For  group,  writes  mean  performance  information  for  this  run  *) 
(ft . . . . . *) 


BEGIN 

FOR  M: =  1  TO  R  DO  BEGIN 
WITH  RUNDATCM)  DO 

WR1TELN  (C1FIL,  RUNSEQNUM: 3 ,  RUNPAR:3,  BLANK,  VTYPE ,  BKGRD: 4 , 
S I GNO I SE : 3 ,  BLANK,  PRECURS)} 

(ft  WRITES  SUBJECT  NUMBER  AND  CORRECT-RESPONSE  FLAGS  *) 

FOR  J  s=  1  TO  S  DO  BEGIN 

WRITE  (C1FIL ,  RUNDATCM). RUNSEQNUM:2,  J:3); 

FOR  I  :=  1  TO  32  DO 

WRITE  (C1FIL,  WARNRESPCM , J , 1 3 : 2) ; 

WRITELN  (C1FIL) 5 
END;  (ft  FOR  J.. .BEGIN  *) 

(ft  WRITES  RESPONSE  TIMES  FOR  EACH  SUBJECT  *) 

FOR  J  :=  1  TO  S  DO  BEGIN 

WRITE  (C1FIL,  RUNDATCM). RUNSEQNUM : 2 f J : 3) ; 

FOR  N  :=  1  TO  10  DO 

WRITE  (C1FIL,  WARNT IMECM , J ,N3 : 7: 3) ; 

WRITELN  (C1FIL) ; 

WRITE  (C1FIL ,  RUNDATCM) . RUNSEQNUMs  2 ,  J:  3) ; 

FOR  N  :=  11  TO  20  DO 

WRITE  (C1FIL ,  WARNT IME CM , J ,N3 : 7: 3) : 

WRITELN  (C1FIL); 

WRITE  (C1FIL,  RUNDATCM). RUNSEQNUM: 2 ,  J:3); 

FOR  N  :=  21  TO  30  DO 

WRITE  (CIFIL,  WARNTIMECM , J ,N3 : 7 : 3) : 

WRITELN  (CIFIL); 

WRITE  (CIFIL,  RUNDATCM). RUNSEQNUM:2.  J :  3) ; 

FOR  N  :=  31  TO  32  DO- 

WRITE  (CIFIL,  WARNTIMECM, J, N3 : 7: 3) ; 

WRITELN  (CIFIL): 

END;  (ft  FOR  J . . , BEG i N  *) 
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(*  WRITES  PERFORMANCE  SUMMARY  FOR  EACH  SUBJECT  IN  THIS  RUN  *> 

FOR  J  : =  1  TO  S  DO  BEGIN 
WITH  RUNDAT CM3  DO 

WRITE  (CiFIL,  RUNSEQNUM:3,  RUNPAR:3,  BLANK,  VTYPE , 

BKGRDs  4,  SIGNOISE: 3,  BLANK,  PRECURS) ; 

WRITELN  (CiFIL,  J:5,  BLTRACC C M , J ] : 6: 3 , 

PRTRACCCM, J3: 6:3,  PRACCLOSS CM . J 3 : 6: 3 , 
PRPERCBL£M,J3:6:3,  WARNCORREM , J 3 : 3 ,  NORESPSUMCM , J ] : 3 , 
WARNACCEM , J 3 : 6: 3 ,  MNRESPTIMECM , J 3 : 6: 3) ; 


END;  (*  FOR  J.. .BEGIN  *) 

(*  WRITES  GROUP  PERFORMANCE  FOR  THIS  RUN  *) 

WITH  RUNDATIM3  DO 

WRITE  (CIFIL,  RUNSEQNUM: 3 ,  RUNPARi 2 ,  SPACER,  VTYPE ,  BKGRD: 4  , 

SIGNOISE: 3 ,  SPACER,  PRECURS); 

WRITELN  (CIFIL,  GPBLTRACCCM3 ; 6: 3 ,  GPPRTRACCCM3 : 6:  3 , 

GPPRACCLOSSCM3 : 6: 3 ,  GPPRPERCBLCM3 : 6:3,  GPMNWARNCORRIM3 : 8: 3, 
GPMNNORESPSUMCM3 ; 6; 3 ,  GPWARNACCEM3:6:3,  GPMNRESPTIMECM3 : 6: 3) 


END;  (*  FOR  M... BEGIN  *) 

END;  UWRITITOUT*) 

(if**************#***********#*##****#***#********#**#********#****#***) 

PROCEDURE  WRITESUBDAT; 


UPSEUDO  CODE  ») 
(*  Write  output  to  "SUBFIL":  *) 
(*  For  each  run:  *) 
(*  Write  run  parameter  information  *) 
(*  For  each  subject,  writes  summary  performance  information  *) 
(* . . . . . *) 


BEGIN 

FOR  M; =  1  TO  R  DO  BEGIN 

FOR  J  :=  1  TO  S  DO  BEGIN 

<*  WRITES  SUBJECT  DATA  TO  "SUPFIL"  *) 

WITH  RUNDAT  CM3  DO 

WRITE  (SUBFIL,  RUNSEQNUM: 3 ,  RUNPAR:3,  BLANK,  VTYPE. 
BKGRD: 4 ,  SIGNOISE:  3 ,  BLANK,  PRECURS);  • 
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WRITELN  (SUBFIL.  J:5,  BLTRACCCM, J 3 : 6: 3 . 

PRTRACCCM , J  3 :  6: 3 ,  PRACCL0SS[M,J3:6:3, 
PRPERCBLCM,Jjs6:3,  WARNCQRRCM, J3 : 3,  NQRESPSUHCM , J 3 : 3 . 
WARNACCCM . J 3 : 6:3,  MNRESPT I  ME C M , J  3 : 6: 3) : 

END;  <*  FOR  J. , , BEGIN  *) 

END;  (*  FOR  M... BEGIN  *) 

END;  <*  WRITESUBDAT  *) 


(*************♦********#*#*#*******#***#**#***************************) 
PROCEDURE  WRITEGPDAT: 


(*PSEUDQ  CODE  *) 
(*  Write  output  to  "GPFIL" :  *) 
(#  For  group,  writes  mean  performance  information  for  each  run  *) 
(* . . . *> 


BEGIN 

FOR  M:  =  1  TO  R  DO  BEGIN 

<*  WRITES  GROUP  DATA  TO  "GPFIL"  *) 

WITH  RUNDATCM3  DO 

WRITE  (GPFIL,  RUNSEQNUM: 3 ,  RUNPAR:2,  SPACER,  VTYPE ,  BKGRD: 4 . 

SIGNOISE: 3,  SPACER,  PRECURS): 

WRITELN  (GPFIL,  GPBLTRACCCM3 : 6: 3 ,  GPPRTR ACC C M 3 : 6:3, 

GPPRACCL0SSCM3 ;  6.-  3 ,  GPPRPERCBLCM3 : 6: 3,  GPMNWARNCORRCM3 :  8:  3 . 
GPMNN0RESPSUMCM3 : 6 : 3 ,  GPWARNACCCM3 :  6:  3 ,  GPMNRESPT I  ME C M 3 : 6:3) 

END;  U  FOR  M... BEGIN  *> 

END;  <*  WRITEGPDAT  *) 

(#**###*■***#####*********#******#*#***#*#*###***************#***##***#) 
PROCEDURE  WRITSDAT; 


(*  PSEUDO  CODE:  *> 
(#  Writes  data  to  "SDAT",  for  use  by  the  SPSS  program  *) 
(*  -*#For  each  run,  do:  •  *) 
(*  #*For  each  subject,  do:  *) 
(*  **For  each  stimulus,  do:  *) 
(*  Write  (one  line  per  stimulus):  *) 
(*  Run  sequence  number  #) 
(*  Voice  type  *) 
(*  Background  engine  noise  level  *> 
(*  Sianal-to-noise  ratio  over  headset  *) 
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<* 

Precursor  type 

*) 

(* 

Subject  number 

*) 

(* 

Stimulus  sequence  number  Cl  to 

31.  in  order! 

*) 

(* 

Stimulus  given  [button  number, 

1  to  313 

*> 

(* 

Response  to  stimulusCl  to  32.  or  99  tor  no  resoonsel  *) 

(* 

Stimulus  response  code  Cl, 0,83 

*) 

(* 

Response  time  for  that  stimulus 

*) 

(* 

Baseline  Primary  task  tracking 

accuracy  for 

this  subj/run*) 

<* 

Primary  individually-normalized 

accuracy  for 

this  sub/run*) 

BEGIN 

FOR  M  :=  1  TO  R  DO  BEGIN  <*  FOR  EACH  RUN  *) 

FOR  J  :=  1  TO  S  DO  BEGIN  <*  FOR  EACH  SUBJECT  *) 

FOR  I  :=  1  TO  32  DO  BEGIN  (*  FOR  EACH  STIMULUS  *) 

STIMNUM  :=  I: 

WITH  RUNDATCM3  DO 

WRITE  CSDAT ,  RUNSEQNUM: 3 ,  SPACER,  VTYPE ,  BKGRD: 4 ,  SIGNOISE: 3, 
SPACER,  PRECURS,  SUBDAT [ J ] . SUBNUM: 3 , 

STIMNUM: 3 ,  STIHULUSC I 3s3) 5 

WRITE  (SDAT ,  STIMRESPCM, J , 1 3 : 3,  WARNRESPCM, J ,13:3, 

WARNTIMECM , J , 13:7:3,  BLTRACCEM , J  3 : 7: 3 , 

PRPERCBLCM . 03:7:3) : 

WRITELN  (SDAT).; 

END:  (*  FOR  I. ..BEGIN  *) 

END:  <*  FOR  J.. .BEGIN  *) 

END:  (*  FOR  M... BEGIN  *) 


END;  (*  WRITSDAT  *) 

(**#***##**#***#**#********###*****#**#*#*##**#♦ a*********************) 


BEGIN 

SETUP: 

READITIN; 

CALCULATE; 

WR I T I  TOUT : 
WRITESUBDAT; 
WRITEGPDAT; 
WRITSDAT; 


END. 


APPENDIX  C 
TTEST1 
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PROGRAM  TTEST1  (INPUT/,  OUTPUT,  SRAWTDT,  TFIL) ; 


<*  READS  "SRAWTDT"  *) 

(*  COMPUTES  T-TE5T  DATA,  AND  OUTPUTS  TO  "TFIL"  *) 

(*===================================================================*) 

(*  NOTE .  *} 

<*  TO  CUSTOMIZE  INPUT,  RESET  THE  CONSTANT  "R"  (NUMER  OF  RUNS)  *> 

(*  IN  THE  "SETUP"  PROCEDURE.  *) 

(*r=r=============. ==================================================♦) 


(*  FOR  FORMAT  OF  INPUT  !<  OUTPUT  FILES,  SEE  "FILE  CONVENTIONS"  FILE  *) 
(a********#*****##****************##*****#****#***************#*******) 


UPSEUDO  CODE:  *) 
(*  Set  up  -files  ciid  variables  #) 
(*  Read  input  CSRAWTOT]  *) 
(*  Do  data  manipulations  .  *) 
(*  **For  each  run,  do:  *) 
(*  *For  each  subject, do:  *) 
(*  Calculate  primary  task  Coveral 1 -basel i ne3  figures  *) 
(*  Calculate  accuracies  [percentages]  *) 
(»  {primary,  overall,  baseline,  reduction,  percent-BL)  *) 
(*  Compare  warning-responses  to  correct  responses  *) 
(*  Assign  "Flag  Codes"  11,0,8]  to  responses  *) 
<*  Create  time-response  array  [matrix]  #) 
(*  Calculate  accuracy  (percent)  and  average  time  *) 
(#  *For  Group,  calculate:  *) 
(*  First  l/3rd  (stimuli  1-11)  accuracy  and  speed  *) 
(*  Last  l/3rd  (stimuli  22-32)  accuracy  and  speed  *) 
(*  ^Perform  grouped  t-test  between  first  and  last  l/3rd  data  *) 
(*  Write  output  CTFIL]  #> 


(*#******#**********#***# *****#**#### ************************* **#***#*) 
CONST 

BLANK  =  •  ’; 

WARNSUM  =  32  j 

S  =  10;  (*  S  =  NUMBER  OF  SUBJECTS  FOR  EACH  RUN  *) 

R  =  54;  (*  R  =  NUMBER  OF  RUNS  IN  SRAWTOT  *) 

TCRIT  =  2.101;  (*  "t"  for  18  d.f.,  F=,975  [2-sided  .05  alpha]  *) 


TYPE 


TINTARRAY  =  ARRAY  C 1 . . R ,  1..S,  1..2]  OF  INTEGER; 
TREALARRAY  =  ARRAY  [1..R,  1..S,  i.,2]  OF  REAL; 
RUNINTARRAY  =  ARRAY  [1..R,  1..23  OF  INTEGER; 
RUNREALARRAY  =  ARRAY  [ 1 . . R,  1..2]  OF  REAL; 
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REACTREC  =  RECORD 

RESPONSE  i  INTEGER; 

RESPTIME  :  REAL; 

END;  UREACTREC*) 

LINEREC  =  RECORD 

LINENUM  ;  INTEGER; 

ANSWER  ;  ARRAY  C1..83  OF  REACTREC; 
END;  ULINEREC*) 

SUBJECTREC  *  RECORD 

FIRSLINENUM  :  INTEGER; 

SUBNUM  ;  INTEGER; 

TRAKTOT  ;  INTEGER; 

TRAKCQRR  :  INTEGER; 

BLTOT  ;  INTEGER; 

BLCORR  :  INTEGER; 

REACTION  :  ARRAY  Cl. .4]  OF  LINEREC; 

END;  USU8JECTREC*) 

TIMEREC  =  RECORD 
YEAR  :  INTEGER; 

MONTH  ;  INTEGER; 

DATE  ;  INTEGER; 

HOUR  ;  INTEGER; 

MINUTE  ;  INTEGER; 

SECOND  ;  INTEGER; 

END;  (*TIMEREC  *) 

RUNREC  =  RECORD 

RECLINENUM  s  INTEGER; 

RUNSEQNUM  ;  INTEGER; 

TIMEDAT  ;  TIMEREC; 

RUNPAR  ;  INTEGER; 

VTYPE  :  CHAR; 

BKGRD  ;  INTEGER; 

SIGNOISE  :  INTEGER; 

PRECURS  ;  CHAR; 

STIMULUS  ;  ARRAY  Cl.. 32]  OF  INTEGER; 
SUBDAT  ;  ARRAY  C 1 . . S3  OF  SUBJECTREC 
END;  (*  RUNREC  *) 


VAR 


SRAWTOT ,  TFIL  :  TEXT; 

G,  H,  I,  J,  K,  L,  M,  N  :  INTEGER; 
STIMNUM  ;  INTEGER; 

WV1,  WV2,  TV1 ,  TV2,  ADJ  ;  REAL; 
DENWARN,  DENTINE  ;  REAL; 
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STIMRESP  :  ARRAY  C1..R,  i..S,  1..323  OF  INTEGER; 

WARNRESP  ;  ARRAY  C1..R,  1..S,  1 . . 323  OF  INTEGER; 

WARNTIME  ;  ARRAY  E1..R,  1..S,  l;.323  OF  REAL; 

RUNDAT  :  ARRAY  C1..R]  OF  RUNREC; 

TWARN  :  ARRAY  U..R3  OF  REAL; 

TTI ME  ;  ARRAY  C1..R3  OF  REAL; 

WARNCORR,  GPWARNSUM  ;  TINTARRAY; 

WARNACC ,  MNRESPTIME,  TIMESUM  ;  TREALARRAY; 

WDIF,  TDIF ,  WDIF3Q,  TDIFSQ  :  TREALARRAY; 
GPWARNACCSUM ,  GPMNWARNACC  ;  RUNREALARRAY; 
GPTIMESUM,  GPMNRESPTIME  :  RUNREALARRAY; 

WARNVAR,  TIMEVAR  ;  RUNREALARRAY; 

SUMWDIFSQ,  SUMTDIFSQ  ;  RUNREALARRAY; 

SPACER  s  CHAR; 


I*******************#************#******##*#****#*********************) 
PROCEDURE  SETUP; 


(#PSEUDQ  CODE;  *) 
(*  Set  up  files'  #) 
(*  Reset  input  files  *) 
(*  Rewrite  output  files  *) 
(#  Initialize  variables  for  summing  *> 
(* . . . *> 


BEGIN 

RESET  (SRAWTQT) ; 

REWRITE  (TFIL) ; 

FOR  M  ;=  1  TO  R  DO  BEGIN 

(*  SUBJECT  SUMS  *)_ 

FOR  I  ;=  1  TO  S  DO  BEGIN 

TIMESUMCM, 1,13  ;=  O.O; 
TIMESUMCM , 1,23  ;=  0.0; 


lib 


WARNCORRCM, 1,13  :«  Oj 
WARNCORRCM , 1,23  :=  0; 

END;  (*FOR  I. ..BEGIN  *> 

(*  GROUP  SUMS  *> 

GPWARNACCSUMCM ,  1  ]  :=  0.0; 

GPWARNACCSUMCM, 23  :=  0.0; 

GPTIMESUMCM , 1 ]  :=  C  0; 

GPTIMESUMCM , 23  :=  0.0; 

SUMWDIFSQCM.13  :=  0.0; 

SUKW0IFSQCM,23  :=  0.0; 

SUMTDIFSQCM, 1 ]  :=  0.0; 

SUMTDIFSQCIi, 2 j  ;=  0.0; 

END;  <*  FOR  M... BEGIN  *) 

END;  <*  SETUP  *) 

(a#****#*****#***********#*#**#***********#*****#***#***#******#******) 
PROCEDURE  READITIN; 

(*PSEUDQ  CODE  *) 


<* 

For  each  run,  do; 

*) 

(* 

Read  record  00 

*) 

(* 

Read  32  "warning"  stimuli  into 

(STIMULUS) 

*) 

(* 

For  each  subject  do 

*) 

(* 

Read  five-record  set  [must 

read  field-by-field] 

*) 

(* 

Format  into  proper  array  information 

*) 

<*. 

BEGIN 

FOR  M  ;=  1  TO  R  DO  BEGIN 
WITH  RUNDATCM3  DO  BEGIN 
READ  (SRAWTOT ,  RECLINENUM,  RUNSEQNUM) ; 

WITH  TIMEDAT  DO 

READ  (SRAWTOT,  YEAR,  MONTH,  DATE,  HOUR,  MINUTE, 

SECOND) ; 

READ  (SRAWTOT,  RUNPAR,  SPACER,  VTYPE ,  BKGRD,  SIGNOISE, 

SPACER,  PRECURS); 

READLN  (SRAWTOT); 

(#  CHECKS  THAT  RUNS  ARE  ENTERED  IN  ORDER  BY  COMPARING  RUNSEQNUM  *) 
{*  WITH  PREVIOUS  RUNSEQNUM,  AND  PRINTS  WARNING  ON  SCREEN  *) 


T 
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-  IF  M  >  i  THEN  BEGIN 
G  :=  M  -1; 

IF  RUNSEQNUM  <>  RUNDATIG3 . RUNSEQNUM  +  1  THEN 

WRITELN  < 'NOTE:  RUN  SEQUENCE',  RUNSEQNUM,'  LISTED  OUT  OF  ORDER'); 
END;  (*  IF  M  *) 

READ  (SRAWTOT,  RECLINENUM); 

FOR  I  :=  1  TO  16  DO 

READ  (SRAWTOT,  STIMULUS! 1 3 ) ; 

READLN  (SRAWTOT); 

READ  (SRAWTOT,  RECLINENUM); 

FOR  I  ;=  17  TO  32  DO 

READ  (SRAWTOT,  STIMULUS!! 3 ) ; 

READLN  (SRAWTOT); 

(*  READ  WARNING  RESPONSE  DATA  FOR  EACH  SUBJECT  *> 

FOR  J  :=  1  TO  S  DO  BEGIN 

N  ;=  1;  (*  RESETS  STIMULUS  NUMBER  *> 

WITH  SUB  DAT ! J  3  DO  BEGIN 

READ  (SRAWTOT,  FIRSLINENUM,  SUBNUM,  TRAKTOT,  TRAKCORR, 

BLTOT,  BLCORR) ; 

READLN  (SRAWTOT); 

FOR  K  s=  1  TO  4- DO  BEGIN 

WITH  REACT  I  ON ! K  3  DO  BEGIN 
READ  (SRAWTOT,  LINENUM); 

FOR  L  ;=  1  TO  S  DO  BEGIN 
WITH  ANSWERIL3  DO  BEGIN 

READ  (SRAWTOT,  RESPONSE,  RESPTIME) ; 

(*  ASSIGNS  DATA  TO  NEW  ARRAYS  *) 

STIMRESPIM, J,N3  ;=  RESPONSE; 

WARNTIMEIM,  J  ,N3  ;=  RESPTIME; 

(*  ASSIGNS  1 “CORRECT ,  0=INCORRECT  OR  MISSING  *) 
IF  RESPONSE  =  STIMULUSCN3  THEN 
WARNRESPIM , J , N3  ;=  1 
ELSE  WARNRESPIM, J,N3  :=  O; 

N  ;=  N  +  1;  <*  UPDATES  WARNING  NUMBER  *) 

END;  (*  WITH  ANSWER. .. BEGIN  *) 

END;  (*  FOR  L... BEGIN  *> 

READLN  (SRAWTOT); 

END;  (*  WITH  REACTON. .. BEGIN  *> 

END;  (*  FOR  K... BEGIN  *> 

END;  (*  WITH  SUBDAT. .. BEGIN  *) 

END;  (*  FOR  J. . .BEGIN  *) 

END;  (*  WITH  RUNDAT. .. BEGIN  *) 

END;  (*  FOR  M... BEGIN  *) 

END;  UREADITIN*) 

(****#******#**************#**************»#**#**********#************) 
PROCEDURE  CALCULATE; 


118 


r 


(♦PSEUDO  CODE 
(*  For  each  run,  do: 

(*  CFor  first-l/3rd  and  last-i/3rd!  calculate: 

(*  For  each  subject: 

(*  Number  of  correct  responses 

(*  Sum  of  response  times  (correct  responses  only) 

(*  Accuracy 

(*  Mean  response  time  (correct  resp's  only) 

(*  [For  accuracy  and  response  times!  : 

(*  Group  means  and  variances  for  first  &  last  l/3rd 

(♦  Gouped  t-test  values 

. . 

BEGIN 

(*  FOR  EACH  RUN  *) 

FOR  M  :=  1  TO  R  DO  BEGIN 

<*  FOR  EACH  SUBJECT:  *) 

FOR  J  :=  1  TO  S  DO  BEGIN 

(*  SUBJECT  SUMS  WERE  SET  TO  ZERO  IN  "SETUP"  *> 

(*  CALCULATES  FIRST-1 /3RD  DATAPOINT  SUMS  *) 

FOR  N  :=  1  TO  11  DO  BEGIN 
CASE  WARNRESPCM, J ,N3  OF 
1  :  BEGIN 

WARNCORRCM ,  J , 1 1  :=  WARNCORRCM,  J  ,  1 3  +  1; 

TIMESUMCM , j , 1 3  :=  TIMESUMEM, J , 13  +  WARNTIMECM . J  , N3 ; 
END;  (*  CASE  1  [CORRECT  RESPONSE!  *> 

Os;  (*  NO  UPDATES  [INCORRECT  RESPONSE!  *) 

8  :  ;  (*  NO  UPDATES  [MISSED  RESPONSE!  *) 

END;  (*  CASE  ♦) 

END;  <♦  FOR  N., .BEGIN  *> 

<*  CALCULATES  LAST-1 /3RD  DATAPOINT  SUMS  *) 

FOR  N  :=  72  TO  32  DO  BEGIN 
CASE  MARNRESPEM ,  J ,  N3  OF 
1  :  BEGIN 

WARNCORREM , J ,23  :=  WARNCORREM , J , 23  +  1; 

TIMESUMCM ,J ,23  :=  TIMESUMEM. J ,23  +  WARNTIMEEM , J , N 3 ; 
END;  <*  CASE  l  [CORRECT  RESPONSE!  *) 

0  :  ;  (*  NO  UPDATES  [INCORRECT  RESPONSE!  *) 

8  :  ;  (*  NO  UPDATES  [MISSED  RESPONSE!  *> 

END;  <*  CASE  *> 

END;  (*  FOR  N... BEGIN  *) 

END;  (*  FOR  J. . .BEGIN  *) 

(*  CALCULATES  MEAN  PERFORMANCE  FOR  FIRST-  AND  LAST-  1/3RD  *) 

FOR  J : =  1  TO  S  DO  BEGIN 


*) 
*> 
*) 
♦  ) 
*) 
*) 
*) 
*) 
*) 
*) 
*) 
*) 


WARNACCCM , J . 1 3  :=  'WARNCORRCM,  J ,13/11: 

'  WARNACCCM, J, 2]  :=  WARNCORRCM, J , 23 / 1 1 ; 

MNRESPTIMECM, J , 1 ]  :=  TIMESUMCM, J , i ] /WARNCORRCM, J ,13; 
MNRESPTIMECM, J ,23  :=  TIMESUMCM, J ,23/WARNCORRCM, J , 23; 

END  j  <*  FOR  J. . .BEGIN  *) 

(#  CALCULATES  GROUP  MEANS  AND  VARIANCES  *) 

<*  CALCULATES  GROUP  SUMS  *) 

FOR  J  :=  1  TO  S  DO  BEGIN 

GPWARNACCSUMCM,  1  ]  :=  GPWARNACCSUHCM , 1 1  +  WARNACCCM, J , 13 
GPWARNACCSUMCM ,23  :=  GPWARNACCSUMCM , 23  +  WARNACCCM, J ,23 

GPTIMESUMCM, 1 3  :=  GPTIMESUMCM,  1 3  +  MNRESPTIMECM, J  ,13; 
GPT IMESUMCM , 23  :=  GPTIMESUMCM, 23  +  MNRESPTIMECM, J ,23; 

END.;  (*  FOR  J. .  .BEGIN  *) 

(*  CALCULATES  GROUP  MEANS  *) 

GPMNWARNACCCM , 1 3  :=  GPWARNACCSUMCM, I  3/S; 

GPMNWARNACCCM,23  :=  GPWARNACCSUMCM, 23/S; 

GPMNRESPTIMECM, 13  :=  GPTIMESUMCM, 1 3/S; 

GPMNRESPTIMECM,23  ;=  GPTIMESUMCM, 23/S; 

(*  CALCULATES  VARIANCES  *) 

FOR  J  i=  1  TO  S  DO  BEGIN 

WDIFCM, J , 1 3  ;=  WARNACCCM . J  ,  13  -  GPMNWARNACCCM . 1 3 ; 
WDIFCM, J ,23  :=  WARNACCCM , J , 23  -  GPMNWARNACCCM, 23; 

TDIFCM , J , 1 3  :=  MNRESPTIMECM, J , 1 3  -  GPMNRESPTIMECM, 1 3; 
TDIFCM, J ,23  i=  MNRESPTIMECM , 3,23  -  GPMNRESPTIMECM, 23 ; 

WDIFSQCM, J  ,  1 3  :=  SQR (WDIFCM , J , 1 3 ) ; 

WDIFSQCM, J ,23  ;=  SQRCWDIFCM, J,23> ; 

TDIFSQCM , 0,13  :=  SQR (TDIFCM ,J ,  1 3 ) ; 

TDIFSQCM, J ,23  :=  SQR (TDIFCM, J ,23 ) ; 

SUMWDIFSQCM, 13  ;=  SUMWDIFSQCM, 1 3  +  WDIFSQCM, J ,13; 
SUMWDIFSQCM,23  !=  SUMWDIFSQCM, 23  +  WD IFSQCM , 0 , 23 ; 

SUMTDIFSQCM , 1 3  ;=  SUMTDIFSQCM, 1 3  +  TDIFSQCM, J , 1 3; 
SUMTDIFSQCM ,2 3  :=  SUMTDIFSQCM, 23  +  TDIFSQCM, J ,23; 

END;  (*  FOR  J. . , BEGIN  *0 
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WARNVARtM, 1 ]  :  =  SUMWDIFSQCM, 1 ] /3j 
Wf  .Nv ARCH, 2 3  :=  SUMWD I FSQC M , 2 ] / S ; 

TIMEVARCM, 1 ]  :=  SUM TDIFSQCM, 13/S; 

TIMEVARCM , 23  ;=  SUMTDIFSQCM , 2D /Sj 

<*  CALCULATES  "T"-VALUE  FOR  T-TEST  *) 

WVi  :  =  WARN VAR  EM,  1 3 ; 

WV2  :=  WARNVARCH.2]; 

TV1  :=  TIMEVARCM, 1]; 

TV2  :=  TIMEVARCM, 21; 

ADJ  s=  (S+S) / (S*S) ; 

DENWARN  :=  SORT (  (  (  (S-1)*WV1  +  (S-1XWV2  )  /  (S+S-2)  )  *  ADJ  ); 
DENTINE  :=  SQRT (  (  (  (S~1)*TV1  +  <S-1)*TV2  )  /  (S+S-2)  )  *  ADJ  ) j 
TWARNCM3  :=  ABS (GPMNWARNACCCM , 1 3-GPMNWARNACCCM , 2] ) /DEN WARN; 
TTIMECM]  :=  ABS ( SPMNRESPT I  ME C M , 1 3 -GPMNRESPT I  ME C M , 2 3  > '/ DENT  I  ME ; 

END;  (*  FOR  M... BEGIN  *) 

END;  (*  CALCULATE  *) 


(#***#***#*#**#*******#*******#******  *■*  *  *******  ***********************) 


PROCEDURE  WRITETFIL; 


<* 

PSEUDO 

CODE; 

*) 

(* 

Writes  results  to  file  "TFIL" 

*) 

(* 

Write  title  lines 

*) 

{* 

For 

each  run  writes 

*) 

(* 

Run  sequence  number 

*) 

(* 

Accuracy  for  first  1 / 3r d 

*) 

(* 

Variance  for  first  l/3rd  accuracy 

*> 

<* 

Accuracy  for  last  1 /3rd 

*) 

(* 

Variance  for  last  l/3rd  accuracy 

*) 

(* 

Mean  response  time  for  first  l/3rd 

*) 

(* 

Variance  for  first  l/3rd  responmse  time 

*> 

(x 

Mean  response  time  for  last  l/3rd 

*) 

(* 

Variance  for  last  l/3rd  response  time 

*) 

(* 

T-value  for  accuracy 

*) 

(* 

T-value  for  response  time 

*) 

(* 

Critical  t-value  for  two-tailed  test,  using 

*) 

<* 

alpha=. 05  and  18  degrees  of  freedom 

*> 

<* 

Indicate  if  difference  is  statistically  significant 

*> 

<*. 

.  .  .  *) 
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BEGIN 


WRITELN  (TFIL,  'T-TEST  FOR  WITHIN-RUN  FATIGUE/LEARNING  CURVE'); 

WRITE  (TFIL,'  RN  ACC-1  VACC1  ACC-2  VACC2  TIME-1  VTIM1  TIME-2  VTIM2 ' ) 
WRITE  (TFIL,  '  TWARN  TTIME  TCRIT  CRIT? ' ) ; 

WRITELN  (TFIL); 

WRITE  (TFIL,'  ==  =====  =====  =====  =====  ======  =====  ======  ====='): 


WRITE  (TFIL,  ' 
WRITELN  (TFIL) 
WRITELN  (TFIL) 


) 


FOR  H  s=  1  TO  R  DO  BEGIN 


WRITE  (TFIL,  RUNDATCM3 . RUNSEQNUMi 3 , 

GPMNWARNACCCM, 13:6:3,  WARNVARCM , 13:6:3, 
GPMNWARNACCCM,23 :6: 3,  WARNVARCM, 23 : 6: 3 . 
GPMNRESPTIMECM, 13:7:3,  TIhEVARCM, 13:6:3, 
GPMNRESPTIMECM, 23 : 7: 3 ,  TIMEVARCM, 23 : 6:3, 
TWARNCM3:6:3,  TTIMECM3 : 6: 3 ,  TCRIT : 6:3); 
IF  TWARNCM3  >=  TCRIT  THEN 
WRITE  (TFIL,  '  W') 

ELSE  WRITE  (TFIL,  '  '); 

IF  TTIMECM3  >=  TCRIT  THEN 
WRITE  (TFIL,  '  T') 

ELSE  WRITE  (TFIL,  '  '); 

'  » 

WRITELN  (TFIL); 

END;  (*FOR  M., .BEGIN  *) 


END;  (*  WRITETFIL  *) 


(a###*************#***#***#***#*****#**##*#************#*#************) 

BEGIN 

SETUP; 

READITIN: 

CALCULATE; 

WRITETFIL; 

END. 
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124 


FILE  CONVENTIONS 


The  -following  are  the  -file  contents  and  -filename 
conventions  used  for  the  experiment: 

1.  RAW01  thru  RAW54 

Raw  data  files,  transferred  directly  from  Hewlett 
Packard  HP9345  into  Apple  lie;  RAW01  is  for  first  "run 
parameter  code",  RAW02  for  second  "run  parameter  code", 
etc.  The  contents  are  explained  under  "SRAWxx"  files. 

2.  TRAW01  thru  TRAW54 

RAWxx  data  files,  after  transforming  them  into  text 
files  for  data  manipulation  and  for  ease  of  transfer 
between  computer  systems  (APPLE  lie  tc  CYBER).  The 
contents  are  explained  under  "SRAWxx"  files. 

3.  SRAW01  thru  SRAW54 

TRAW  data  files,  after  the  following  transf ormations: 

a.  The  Run-sequence-number  was  added  to  the 
beginning  of  each  file  to  indicate  the  order  of  the  trial, 

b.  Commas  and  colons  used  as  separators  in  the 
RAWxx  and  TRAWxx  files  were  transformed  into  spaces,  and 

c.  Ordering  of  the  files  was  changed,  so  that 

SRAW01  corresponds  to  Run-Sequence-number  1  (e.g.  TRAW01 

was  for  run-parameter  1,  which  was  the  eighth  trial  run; 
therefore  SRAW08  contains  the  same  data  as  TRAW01.  Changes 
in  the  file  numbering  scheme  were  necessitated  in  order  not 
to  exceed  USAF  noise  exposure  limitations.). 

The  first  number  of  each  record  is  four  (4)  digits; 
the  first  two  digits  indicate  the  run  parameter  code 
(leading  zero  is  not  printed) ,  the  last  two  indicate  the 
record  number. 

Each  of  the  files  contain  the  following  information: 
Record  00: 

Run  sequence  number  C01  thru  543 
Date  of  run  CYY:MM:DD  (Year: Mon th : Date) 3 
Time  run  started  CHH:MI:SS  (Hour: Minute: Second) 3 
Run  parameter  code  C01  thru  543 
Voice  type  CM  (male),  F  (female),  N  (machine) 3 
Background  noise  level  C105,  1153  in  db 
Signal-to-Noise  ratio  over  headset  CO,  5,  103  in  db 
Precursor  type  CT  (tone),  V  (voice),  R  (repeated 
warning) 3 

Records  01-02: 

Warning  stimulus  crder  C01  thru  32  (nonsequential ,  in 
the  order  presented) 3 


The  -following  -five-record  sets  are  repeated  -for  each  o-f 
the  ten  subjects: 

First  Record: 

Subject  number  C01  thru  103 

Number  o-f  total  tracking  (primary)  task  stimuli 
Number  o-f  correct  responses  to  tracking  (primary)  task 
Number  c-f  baseline  tracking  (primary)  task  stimuli 
Number  o-f  baseline  correct  responses  to  tracking 
(primary)  task 

Second  thru  Fi-fth  Records: 

Thirty-two  sets  o-f  "Response/Time"  to  the  warning 
stimuli,  indicating: 

a.  Responses  to  warning  stimuli  C01  thru  32  (in  the 
order  received;  99  indicates  no  reponse  in  the  time 
allotted;  correct  responses  should  match  records  01-02)3, 
and 

b.  Response  time  Cin  seconds3  -for  the  preceedina 
warning- 

Records  03-07  are  -for  subject  1; 

Records  08-12  are  -for  subject  2; 

Records  13-17  are  -for  subject  3; 

etc. ,  thru 

Records  48-52  are  -for  subject  10. 

4.  SRAWT0T 

Files  SRAW01  thru  SRAW54  combined  into  1  giant 
data-f  i  le. 

5. '  C101  thru  C154 

Conversion  1  o-f  SRAW01  thru  SRAW54. 

Title  Record: 

Run  Sequence  Number,  Run  Parameter  Code,  Voice  type, 
Background  noise,  S/N  ratio,  Precursor  type 
The  -following  -four — record  sets  are  repeated  -for  each  o-f 
the  ten  subjects: 

First  Record: 

Subject  number 

Response-Flags  to  warning  stimuli  Cl  (correct) ,  0 
(incorrect) ,  8  (No  response  in  allotted  time) 3 

Second  thru  Fourth  Records  (-first  number  on  each  record 
is  subject  number): 

Response  times  -for  each  of  the  above  warning  stimuli 
Records  01-04  are  for  subject  1; 

Records  05-08  are  for  subject  2; 
etc. ,  thru 

Records  37-40  are  for  subject  10. 

Records  41  thru  50  (one  record  per  subject) : 

Run  sequence  number 
Run  parameter  code 
Voice  type 

Background  engine  noise 
S/N  ratio  over  headset 
Precursor  type 
Subject  number 


Baseline  primary  task  performance  accuracy  (percent 
correct  reponses  during  baseline  period) 

Primary  task  performance  accuracy  (percent  correct 
responses,  excluding  reponses  during  baseline  period) 

Primary  performance  reduction  (baseline  minus  primary) 
Primary  percentage  of  peak  Cbaselinel  performance 
(primary  over  baseline) 

Secondary  (warning)  task  number  correct  reponses 
Secondary  task  number  of  "no-responses  within  allotted 

time" 

Secondary  task  performance  (percent  correct  responses) 
Secondary  task  average  response  time  for  correct 
responses 

Record  51: 

(Group  performance  measures): 

Runs  sequence  number 
Run  parameters  code 

Baseline  primary  task  performance  accuracy 
Primary  task  performance  accuracy 
Primary  task  performance  reduction 
Primary  task  percentage  reduction 
Secondary  task  mean  number  correct 

‘Secondary  task  mean  number  of  "no-responses  within 
allotted  time" 

Secondary  task  overall  performance  accuracy 
Secondary  task  overall  reaction  speed  (correct 
reponses  only) 

6.  C1FIL 

Files  C101  thru  C154  combined  into  1  giant  datafile, 
including  group  performance  measures. 

7.  SUBFIL 

Subject  summary  measures,  as  described  in  records 
41-50  for  each  "Clxx"  file:  one  record  per  subject,  for 
each  run.  Used  as  data  base  for  some  regression  runs  and 
the  MANOVA. 

8.  GPFIL 

Group  performance  measures  only,  as  listed  in  Record 
51  for  each  "Clxx"  file;  one  record  for  each  run. 

9.  SDAT 

Data  file  for  use  with  some  regressions,  the  ANOVA 
program,  the  within-runs  t-test,  and  other  programs.  The 
data  file  is  organized  in  order  of  the  sequence  of  runs; 
within  each  run,  by  subject  number;  within  each  subject,  by 
the  sequence  in  which  the  stimuli  were  given. 

Each  line  (record)  contains  the  following  information: 

Run  sequence  number  Cl -54 3 
Voice  type  CM,  F,  N3 

Background  engine  noise  level  C105,  1153  in  db 
Signal-to-noise  ratio  over  headset  CO,  5,  103  in  db 
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Precursor  type  CT,  W,  R3 
Subject  number  C 1-103 

Stimulus  sequence  number  Cl— 32,  in  order! 

Stimulus  El-32,  corresponds  to  stimulus  actually 
called  -for] 

Response  to  stimulus  Cl-32,  or  99  -for  no  response! 
Stimulus  response  code  Cl,  0,  S3 
Response  time  -for  stimulus,  in  seconds 
Basel in  primary  task  tracking  accuracy 
Primary  task  individually-normalized  accuracy  during 
"stimulus"  conditions. 
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^F‘F=*e:t^ir>  i  x  r>s 

SUBJECT  PERFORMANCE  FILE 
<SUBFIL) 


RN  RP  V  BKD  SN  PC  SUB  BLINE  PRIM  ACLOS  NQRML  CR  NR  ACCY  TIME 


1 

5 

N 

115 

0 

W 

1 

0.942 

0.912 

0.030 

0.968 

29 

1 

0.906 

3. 125 

1 

5 

N 

115 

0 

w 

2 

0.912 

0.852 

0.060 

0.934 

26 

3 

0.812 

3.332 

1 

5 

N 

115 

0 

w 

3 

0.826 

0.771 

0.055 

0.933 

11 

4 

0.344 

3.057 

1 

5 

N 

115 

0 

w 

4 

0.900 

0.877 

0.023 

0.974 

25 

2 

0.781 

2.408 

1 

5 

N 

115 

0 

w 

5 

0.758 

0.614 

0.145 

0.809 

26 

1 

0.812 

2.969 

1 

4 

5 

N 

115 

0 

w 

b 

0.862 

0.862 

0.001 

0.999 

29 

2 

0.906 

2.727 

1 

5 

N 

115 

0 

w 

7 

0.966 

0.926 

0.040 

0.958 

28 

0 

0.875 

2.606 

1 

5 

N 

115 

0 

M 

8 

0.800 

0.765 

0.035 

0.956 

27 

2 

0.844 

2.847 

1 

5 

N 

115 

0 

w 

9 

0.731 

0.614 

0.116 

0.841 

27 

0 

0.844 

2.498 

1 

5 

N 

115 

0 

w 

10 

0.948 

0.911 

0.037 

0.960 

29 

1 

0.906 

2.093 

2 

6 

M 

115 

10 

T 

1 

0.920 

0.912 

0.009 

0.990 

32 

0 

1.000 

2.902 

2 

6 

M 

115 

10 

T 

2 

0.817 

0.813 

0.004 

0.995 

29 

3 

0.906 

3.202 

2 

6 

M 

115 

10 

T 

3 

0.738 

0.612 

0.125 

0.830 

25 

0 

0.781 

2.827 

2 

6 

M 

115 

10 

T 

4 

0.893 

0.814 

0.079 

0.912 

30 

1 

0.937 

2. 126 

2 

6 

M 

115 

10 

T 

5 

0.809 

0.761 

0.049 

0.940 

32 

0 

1 . 000 

2.782 

2 

6 

M 

115 

10 

T 

6 

0.909 

0.864 

0.045 

0.951 

32 

0 

1.000 

2.540 

n 

<C 

b 

M 

115 

10 

T 

7 

0.958 

0.934 

0.024 

0.975 

32 

0 

1 . 000 

2.395 

2 

b 

M 

115 

10 

T 

8 

0.767 

0.692 

0.076 

0.901 

31 

0 

0.969 

2.701 

2 

b 

M 

115 

10 

T 

1 

9 

0.805 

0.718 

0.087 

0.892 

30 

0 

0.937 

2.290 

2 

b 

M 

115 

10 

T 

10 

0.938 

0.925 

0.014 

0.985 

32 

0 

1.000 

1.889 

3 

13 

M 

105 

10 

T 

1 

0.920 

0.909 

0.011 

0.988 

32 

0 

1.000 

2.786 

3 

13 

M 

105 

10 

T 

2 

0.831 

0.763 

0.068 

0.919 

32 

o 

1.000 

3.  146 

3 

13 

M 

105 

10 

T 

3 

0.771 

0.747 

0.024 

0.968 

23 

0 

0.719 

2.748 

3 

13 

M 

105 

10 

T 

4 

0.938 

0.903 

0.035 

0.963 

30 

0 

0.937 

2.201 

3 

13 

M 

105 

10 

T 

5 

0.895 

0.864 

0.030 

0.966 

29 

1 

0.906 

3.030 

3 

13 

M 

105 

10 

T 

6 

0.917 

0.856 

.  0.060 

0.934 

30 

0 

0.937 

2.493 

3 

13 

M 

105 

10 

T 

7 

0.928 

0.924 

0.004 

0.996 

30 

0 

0.937 

2.403 

3 

13 

M 

105 

10 

T 

8 

0.817 

0.766 

0.051 

0.938 

31 

0 

0.969 

2.532 

3 

13 

M 

105 

10 

T 

9 

0.674 

0.696 

-0.022 

1.032 

29 

0 

0.906 

2.256 

3 

13 

M 

105 

10 

T 

10 

0.907 

0.905 

0.002 

0.998 

32 

0 

1.000 

1.821 

4 

10 

F 

115 

10 

R 

1 

0.952 

0.937 

0.015 

0.984 

32 

0 

1 . 000 

2.813 

4 

10 

F 

115 

10 

R 

2 

0.902 

0.843 

0.059 

0.934 

32 

0 

1.000 

3.254 

4 

10 

F 

115 

10 

R 

3 

0.884 

0.792 

0.092 

0.896 

23 

0 

0.719 

2.635 

4 

10 

F 

115 

10 

R 

4 

0.924 

0.876 

0.048 

0.948 

32 

0 

1.000 

2.040 

4 

10 

F 

115 

10 

R 

5 

0.795 

0.783 

0.011 

0.986 

32 

0 

1 . 000 

•2.835 

4 

10 

F 

115 

10 

R 

b 

0,898 

0.844 

0.055 

0.939 

32 

0 

1.000 

2.226 

4 

10 

F 

115 

10 

R 

7 

0.926 

0.937 

-0,010 

1.011 

28 

1 

0.875 

2.887 

4 

10 

F 

115 

10 

R 

8 

0.813 

0.753 

0.060 

0.927 

31 

0 

0.969 

2.475 

4 

10 

F 

115 

10 

R 

9 

0.635 

0.626 

0.010 

0.984 

29 

o 

0.906 

2.115 

4 

10 

F 

115 

10 

R 

10 

0.962 

0.908 

0.054 

0.944 

31 

0 

0.969 

1,779 

5 

9 

M 

115 

5 

W 

1 

0.924 

0.930 

-0.005 

1.006 

32 

0 

1.000 

3,095 

5 

9 

M 

115 

5 

W 

2 

0.851 

0.803 

0.048 

0.943 

30 

1 

0.937 

3.448 

5 

9 

M 

115 

5 

W 

3 

0.746 

0.591 

0.155 

0.793 

26 

0 

0.812 

2.970 

5 

9 

M 

115 

5 

W 

4 

0.903 

0.840 

0.063 

0.930 

31 

l 

0.969 

2.349 

5 

9 

M 

115 

5 

W 

5 

0.879 

0.810 

0.069 

0.922 

29 

2 

0.906 

3.  158 

5 

9 

M 

115 

5 

W 

b 

0.851 

0.839 

0.012 

0.986 

29 

1 

0.906 

2.620 

5 

9 

M 

115 

5 

W 

7 

0.942 

0.928 

0.014 

0.985 

29 

0 

0.906 

2.872 

5 

9 

M 

115 

5 

w 

8 

0.809 

0.793 

0.016 

0.980 

29 

1 

0.906 

2.783 

5 

9 

M 

115 

5 

w 

9 

0.712 

0.625 

0.086 

0.879 

29 

0 

0.906 

2.736 

5 

9 

M 

115 

5 

w 

10 

0.952 

0.916 

0.036 

0.962 

30 

0 

0.937 

2.  105 
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RN 

RP 

V 

BKD 

SN 

PC 

SUB 

8LINE 

PRIM 

ACLOS 

NORML 

CR 

NR 

ACCY 

TIME 

6 

11 

N 

105 

0 

T 

1 

0.928 

0.906 

0.022 

0.976 

29 

0 

0.906 

3.337 

6 

11 

N 

105 

0 

T 

2 

0.839 

0.784 

0.055 

0.934 

25 

2 

0.781 

3.305 

6 

11 

N 

105 

0 

T 

3 

0.724 

0.528 

0.195 

0.730 

26 

l 

0.812 

2.943 

6 

11 

N 

105 

0 

T 

4 

0.877 

0.824 

0.052 

0.940 

28 

1 

0.875 

2.231 

6 

11 

N 

105 

0 

T 

5 

0.863 

0.842 

0.021 

0.975 

28 

1 

0.875 

2.725 

<b 

11 

N 

105 

0 

T 

b 

0.980 

0.895 

0.085 

0.913 

28 

2 

0.875 

2.483 

6 

11 

N 

105 

0 

T 

7 

0.946 

0.902 

0.045 

0.953 

31 

0 

0.969 

2.681 

6 

11 

N 

105 

0 

T 

8 

0.809 

0.715 

0.095 

0  ■  88o 

28 

0 

0.875 

2.618 

b 

11 

N 

105 

0 

T 

9 

0.771 

0.604 

0.  167 

0.783 

28 

0 

0.875 

2.563 

b 

11 

N 

105 

0 

T 

10 

0.964 

0.910 

0,054 

0.944 

30 

0 

0.937 

2.041 

1 

14 

F 

105 

10 

W 

1 

0.903 

0.899 

0.003 

0.996 

32 

0 

1.000 

3.090 

1 

14 

F 

105 

10 

W 

2 

0.811 

0.802 

0.009 

0.989 

31 

0 

0.969 

3.562 

1 

14 

F 

105 

10 

w 

3 

0.471 

0.545 

-0.074 

1.157 

24 

0 

0.750 

2.808 

7 

14 

F 

105 

10 

M 

4 

0.905 

0.833 

0.072 

0.921 

31 

0 

0.969 

2.276 

7 

14 

F 

105 

10 

W 

5 

0.781 

0.830 

-0.049 

1.063 

32 

0 

1.000 

3.024 

7 

14 

F 

105 

10 

W 

6 

0.934 

0.876 

0.058 

0.938 

32 

0 

1.000 

2.660 

7 

14 

F 

105 

10 

W 

7 

0.932 

0.920 

0.013 

0.987 

31 

1 

0.969 

2.760 

7 

14 

F 

105 

10 

W 

8 

0.694 

0.747 

-0. 053 

1.076 

30 

1 

0.937 

2.365 

7 

14 

F 

105 

10 

w 

9 

0.726 

0.727 

-0.001 

1.002 

32 

0 

1 . 000 

2.747 

7 

14 

F 

105 

10 

w 

10 

0.922 

0.901 

0.022 

0.976 

32 

0 

1.000 

2.  100 

8 

1 

M 

115 

0 

R 

1 

0.928 

0.915 

0.013 

0.986 

30 

0 

0.937 

3.108 

8 

1 

n 

115 

0 

R 

2 

0.899 

0.862 

0.037 

0.959 

32 

0 

1.000 

3.020 

8 

1 

M 

115 

0 

R 

3 

0.829 

0.845 

-0.016 

1.019 

23 

0 

0.719 

3.252 

8 

1 

M 

115 

0 

R 

4 

0.831 

0.841 

-0.010 

1.012 

30 

0 

0.937 

2.357 

8 

1 

M 

113 

0 

R 

5 

0.885 

0.842 

0.043 

0.951 

32 

0 

1.000 

2.719 

8 

1 

M 

115 

0 

R 

6 

0.952 

0.928 

0.025 

0.974 

32 

0 

1 . 000 

2.440 

8 

1 

11 

115 

0 

R 

7 

0.982 

0.952 

0.030 

0.969 

31 

0 

0.969 

2.909 

8 

1 

11 

115 

0 

R 

8 

0.473 

0.658 

-0.185 

1.392 

30 

0 

0,937 

2.746 

8 

1 

M 

115 

0 

R 

9 

0.767 

0.712 

0.055 

0.928 

32 

0 

1.000 

2.647 

8 

1 

M 

115 

0 

R 

10 

0.940 

0.900 

0.040 

0.958 

32 

0 

1.000 

1.974 

9 

2 

N 

115 

5 

T 

1 

0.916 

0.905 

0.011 

0.988 

30 

0 

0.937 

3.113 

9 

2 

N 

115 

5 

T 

2 

0.870 

0.841 

0.029 

0.967 

26 

1 

0.812 

2.966 

9 

2 

N 

115 

5 

T 

3 

0,780 

0.702 

0.078 

0.900 

17 

1 

0.531 

3.  197 

9 

2 

N 

115 

5 

T 

4 

0.942 

0.905 

0.037 

0.960 

27 

3 

0.844 

2.398 

9 

2 

N 

115 

5 

T 

5 

0.898 

0.855 

0.043 

0.952 

29 

0 

0.906 

2.885 

9 

2 

N 

115 

5 

T 

6 

0,928 

0.895 

0.033 

0.964 

28 

2 

0.875 

2.543 

9 

2 

N 

115 

5 

T 

7 

0.968 

0.947 

0.021 

0.978 

32 

0 

1.000 

2.970 

9 

2 

N 

115 

5 

T 

8 

0.683 

0.695 

-0.012 

1.018 

28 

0 

0.875 

3.071 

9 

2 

N 

115 

5 

T 

9 

0.804 

0.748 

0.057 

0.929 

29 

0 

0.906 

2.698 

9 

2 

N 

115 

5 

T 

10 

0.932 

0.900 

0.032 

0.965 

29 

0 

0.906 

1.922 

10 

3 

N 

105 

0 

W 

1 

0.936 

0.906 

0.030 

0.968 

31 

1 

0.969 

3.  182 

10 

3 

N 

105 

0 

W 

2 

0.900 

0.846 

0.055 

0.939 

28 

2 

0.875 

3.287 

10 

3 

N 

105 

0 

W 

3 

0.837 

0.614 

0.222 

0.734 

31 

0 

0.969 

3. 230 

10 

3 

N 

105 

0 

w 

4 

0.920 

0.810 

0.111 

0.880 

29 

1 

0,906 

2.509 

10 

3 

N 

105 

0 

w 

5 

0.880 

0.849 

0.031 

0.964 

30 

1 

0.937 

3.034 

10 

3 

N 

105 

0 

w 

b 

0.906 

0.854 

0.053 

0.942 

29 

2 

0.906 

2.696 

10 

3 

N 

105 

0 

w 

T 

/ 

0.972 

0.937 

0,035 

0.964 

32 

0 

1.000 

3. 067 

10 

3 

N 

105 

0 

w 

8 

0.801 

0.616 

0.  185 

0.769 

27 

2 

0.844 

3 . 003 

10 

3 

N 

105 

0 

w 

9 

0.865 

0.710 

0,155 

0.821 

28 

0 

0.875 

2.719 

10 

3 

N 

105 

0 

w 

10 

0.926 

0.903 

0.023 

0.975 

31 

0 

0.969 

2.044 

131 
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RN 

RP 

V 

BKD 

SN 

PC 

SUB 

8LINE 

PRIti 

ACLOS 

NORML 

CR 

NR 

ACCY 

TIME 

11 

15 

F 

105 

10 

R 

1 

0.908 

0.916 

-0.008 

1.009 

31 

0 

0.969 

3.293 

11 

15 

F 

105 

10 

R 

2 

0  •  8  o  0 

0.802 

0.028 

0.966 

31 

0 

0.969 

3.550 

11 

15 

F 

105 

10 

R 

3 

0.758 

0.748 

0.010 

0.986 

27 

0 

0.844 

2.843 

11 

15 

F 

105 

10 

R 

4 

0.872 

0.840 

0.  033 

0.963 

32 

0 

1.000 

2.211 

11 

15 

F 

105 

10 

R 

5 

0.858 

0.838 

0.021 

0.976 

32 

0 

1.000 

2.882 

11 

15 

F 

105 

10 

R 

6 

0.794 

0.820 

-0.025 

1.032 

32 

0 

1.000 

2.259 

11 

15 

F 

105 

10 

R 

7 

0.972 

0.943 

0.029 

0.970 

32 

0 

1 . 000 

2.703 

11 

15 

F 

105 

10 

R 

8 

0.591 

0.612 

-0.021 

1.035 

32 

0 

1 . 000 

2.737 

11 

15 

F 

105 

10 

R 

9 

0.842 

0.823 

0.019 

0.977 

31 

0 

0.969 

2.602 

11 

15 

F 

105 

10 

R 

10 

0.908 

0.894 

0.014 

0.984 

32 

0 

1.000 

1.845 

12 

4 

N 

115 

0 

T 
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3 

0.844 

2.739 

51 

51 

M 

115 

10 

w 

9 

0.832 

0.811 

0.021 

0.975 

30 

0 

0.937 

2.997 

51 

51 

M 

115 

10 

w 

10 

0.934 

0.933 

0.001 

0.999 

27 

0 

0.844 

2.394 

52 

52 

M 

105 

5 

R 

1 

0.930 

0.938 

-0.008 

1.008 

31 

1 

0.969 

3.441 

52 

52 

M 

105 

5 

R 

2 

0.8P9 

0.847 

0.042 

0.953 

31 

0 

0.969 

3.011 

52 

52 

M 

105 

5 

R 

3 

0.875 

0.848 

0.027 

0.970 

30 

0 

0.937 

3.597 

52 

52 

M 

105 

5 

R 

4 

0.954 

0.926 

0.028 

0.971 

31 

1 

0.969 

2.135 

52 

52 

M 

105 

5 

R 

5 

0.922 

0.876 

0.047 

0.950 

32 

0 

1.000 

2.959 

52 

52 

M 

105 

5 

R 

6 

0.861 

0.  873 

-0.012 

1.014 

32 

0 

1.000 

2.736 

52 

52 

M 

105 

5 

R 

1 

0.972 

0.967 

0.005 

0.994 

32 

0 

1 . 000 

3.202 

52 

52 

M 

105 

5 

R 

8 

0.966 

0.869 

0.098 

0.899 

26 

6 

0.812 

2.425 

52 

52 

M 

105 

5 

R 

9 

0.865 

0.790 

0.075 

0.914 

32 

0 

1.000 

2.447 

52 

52 

M 

105 

5 

R 

10 

0.934 

0.916 

0.019 

0.980 

32 

0 

1 . 000 

2.396 

53 

53 

M 

105 

0 

H 

1 

0.946  ’ 

0.932 

0.014 

0.985 

31 

0 

0.969 

3.345 

53 

53 

H 

105 

0 

W 

2 

0.920 

0.854 

0.066 

0.928 

30 

0 

0.937 

2.899 

53 

53 

M 

105 

0 

W 

3 

0.835 

0.826 

0.009 

0.989 

29 

1 

0.906 

3.450 

53 

53 

M 

105 

0 

W 

4 

0.948 

0.923 

0.025 

0.973 

29 

2 

0.906 

2.477 

53 

53 

M 

105 

0 

W 

5 

0.909 

0.827 

0.082 

0.910 

29 

0 

0.906 

2.922 

53 

53 

M 

105 

0 

W 

b 

0.887 

0.817 

0.069 

0.922 

29 

1 

0.906 

2.905 

53 

53 

M 

105 

0 

M 

7 

0.964 

0.971 

-0.007 

1.007 

31 

0 

0.969 

2.969 

53 

53 

M 

105 

0 

W 

8 

0.950 

0.908 

0.043 

0.955 

25 

4 

0.781 

2.760 

53 

53 

M 

105 

0 

W 

9 

0.797 

0.727 

0.070 

0.912 

30 

0 

0.937 

2.643 

53 

53 

M 

105 

0 

W 

10 

0.917 

0.914 

0.003 

0.997 

27 

0 

0.844 

2.409 

54 

54 

F 

105 

0 

R 

1 

0.958 

0.928 

0,030 

0.969 

31 

0 

0.969 

3.285 

54 

54 

F 

105 

0 

R 

2 

0.817 

0.858 

-0.041 

1.050 

29 

1 

0.906 

2.645 

54 

54 

F 

105 

0 

R 

3 

0. 833 

0,841 

-0.008 

1.010 

29 

0 

0.906 

3.441 

54 

54 

F 

105 

0 

R 

4 

0.950 

0.937 

0.013 

0.986 

29 

1 

0.906 

2.143 

54 

54 

F 

105 

0 

R 

5 

0.839 

0.871 

-0.032 

1.039 

32 

0 

1.000 

2.828 

54 

54 

F 

105 

0 

R 

6 

0.823 

0.885 

-0.062 

1.075 

28 

1 

0,875 

2.507 

54 

54 

F 

105 

0 

R 

7 

0.970 

0.970 

0,000 

1.000 

32 

0 

1 . 000 

3.  184 

54 

54 

F 

105 

0 

R 

8 

0.877 

0. 883 

-0.006 

1.007 

26 

5 

0.812 

2.519 

54 

54 

F 

105 

0 

R 

9 

0.734 

0.753 

-0.020 

1.027 

31 

0 

0.969 

2.  142 

54 

54 

F 

105 

0 

R 

10 

0.932 

0.935 

-0.002 

1.003 

32 

0 

1.000 

2.221 

140 
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appendix  r>3 
GROUP  PERFORMANCE  FILE 


(GPFIL) 


RN 

RP 

V 

8KD 

SN 

P 

BLINE 

PRIM 

ACLOS 

MORML 

MNCORR 

NQRSP 

ACCY 

TIME 

1 

5 

N 

115 

0 

W 

0.865 

0.810 

0.054 

0.937 

25.700 

1.600 

0.303 

2.745 

2 

6 

M 

115 

10 

T 

0.855 

0.804 

0.051 

0.940 

30.500 

0.400 

0.953 

2.557 

3 

13 

M 

105 

10 

T 

0.860 

0.833 

0.026 

0.969 

29.800 

0.  100 

0.931 

2.537 

4 

10 

F 

115 

10 

R 

0.869 

0.830 

0.039 

0.955 

30.200 

0. 100 

0.944 

2.503 

5 

<? 

M 

115 

5 

1*1 

0.857 

0.808 

0.049 

0.942 

29.400 

0.600 

0.919 

2.811 

6 

11 

N 

105 

0 

T 

0.870 

0.791 

0.079 

0.909 

28. 100 

0.700 

0.878 

2.682 

7 

1-4 

F 

105 

10 

W 

0.808 

0.808 

0.000 

1.000 

30.700 

0.200 

0.959 

2.783 

8 

1 

M 

115 

0 

R 

0.849 

0.845 

0.003 

0.996 

30.400 

0.000 

0.950 

2.700 

9 

2 

N 

115 

5 

T 

0.872 

0.839 

0.  033 

0.962 

27.500 

0.700 

0.859 

2.762 

10 

3 

N 

105 

0 

1*1 

0.894 

0.805 

0.090 

0.899 

29.600 

0.900 

0.925 

2.877 

11 

15 

F 

105 

10 

R 

0.834 

0.824 

0.010 

0.988 

31 . 200 

0.000 

0.975 

2 . 686 

12 

4 

N 

115 

0 

T 

0.900 

0.870 

0.030 

0.967 

29. 100 

0.900 

0.909 

2.816 

13 

7 

tl 

115 

5 

T 

0.857 

0.857 

0.000 

1.000 

29.800 

0.600 

0.931 

2.852 

14 

16 

F 

105 

5 

R 

0.856 

0.848 

0.008 

0.991 

30.600 

0.300 

0.956 

2.849 

15 

17 

F 

105 

0 

T 

0.883 

0.861 

0.023 

0.974 

30. 600' 

0.600 

0.956 

2.666 

16 

8 

M 

115 

0 

W 

0.918 

0.849 

0.070 

0.924 

30.000 

0.300 

0.937 

2.979 

17 

12 

F 

115 

5 

T 

0.905 

0.839 

0.066 

0.927 

29.400 

0.800 

0.919 

2.743 

18 

18 

M 

105 

0 

R 

0,911 

0.846 

0.065 

0.928 

31.200 

0.400 

0.975 

2.958 

19 

19 

N 

105 

C 

R 

0.896 

0.859 

0.037 

0.959 

31.200 

0.500 

0.975 

3.030 

20 

20 

F 

115 

0 

11 

0.936 

0.878 

0.058 

0.938 

26.700 

0.800 

0.834 

3.017 

21 

21 

N 

105 

10 

R 

0.914 

0.865 

0,049 

0.946 

30.800 

0.500 

0.962 

3.144 

22 

22 

F 

105 

0 

W 

0.915 

0.856 

0.059 

0.936 

30.000 

0.400 

0.937 

2.975 

23 

24 

F 

115 

5 

W 

0. 836 

0.852- 

-0.016 

1.019 

28.700 

0.800 

0.897 

2.876 

24 

23 

M 

105 

5 

T 

0.903 

0.876 

0.027 

0.970 

30.700 

0.300 

0.959 

2.760 

25 

25 

M 

115 

10 

R 

0.915 

0.850 

0.066 

0.928 

31 . 300 

0.  100 

0.978 

3.020 

26 

26 

N 

105 

10 

T 

0.883 

0.834 

0.050 

0.944 

29.900 

0.500 

0.934 

2.722 

27 

27 

F 

115 

0 

R 

0.909 

0 . 838 

0.071 

0.922 

29.400 

0.200 

0.919 

2.694 

28 

23 

N 

115 

10 

T 

0.943 

0.885 

0.057 

0.939 

30. 300 

0.300 

0.947 

2.721 

29 

29 

N 

115 

5 

R 

0.904 

0 . 866 

0.038 

0.958 

30.800 

0.300 

0.962 

2.907 

30 

32 

11 

105 

10 

R 

0.893 

0.836 

0.057 

0.936 

31 .500 

0.200 

0.984 

2.917 

31 

30 

F 

115 

0 

T 

0.911 

0.880 

0.031 

0.966 

29.700 

0.600 

0.928 

2.712 

32 

3 1 

M 

115 

5 

R 

0.893 

0.875 

0.017 

0.981 

31 . 300 

0.500 

0.978 

2.937 

33 

35 

11 

105 

10 

W 

0.969 

0.830 

0.038 

0.956 

30 . 300 

0.700 

0.947 

2.974 

34 

37 

F 

105 

10 

T 

0.871 

0.826 

0.046 

0.  948 

30.300 

0.700 

0.947 

2.667 

35 

33 

N 

115 

5 

W 

0.932 

0.905 

0.027 

0.971 

29.400 

0.700 

0.919 

2.961 

36 

34 

N 

115 

10 

R 

0.930 

0. 883 

0.048 

0.949 

30.900 

0.600 

0.966 

2.973 

37 

39 

M 

105 

0 

T 

0.853 

0.880- 

-0.027 

1.031 

29.200 

0.900 

0.913 

2.693 

38 

41 

F 

105 

5 

1*1 

0.907 

0.845 

0.062 

0.931 

30.300 

0.700 

0.947 

2.788 

39 

36 

M 

115 

0 

T 

0.937 

0.885 

0.052 

0.945 

28.700 

0.900 

0.897 

2.661 

40 

38 

F 

115 

10 

W 

0.889 

0.885 

0.004 

0.996 

29.300 

0.800 

0.916 

2.806 

41 

42 

F 

105 

5 

T 

0.  853 

0.850 

0.002 

0.997 

29.600 

1.300 

0.925 

2.583 

42 

43 

N 

105 

5 

T 

0.836 

0.838- 

■0.002 

1 . 003 

30.000 

1.000 

0.937 

2,663 

43 

44 

11 

105 

5 

W 

0.944 

0.884 

0.060 

0.937 

30.400 

0.800 

0.950 

2.761 

44 

45 

N 

115 

10 

U 

0.910 

0.889 

0.021 

0.977 

28.900 

1.000 

0.903 

2.964 

45 

40 

F 

115 

5 

R 

0.840 

0.856- 

-0.016 

1.019 

29. 100 

1.200 

0.909 

2.969 

142 
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RN  RP  V  BKG  SN  P  P-BLA  P-ACC  ACLSS  PRCBL  MNCQRR  NORSP  ACC'Y  TIME 


46  48  N  105  10  U  0,923  0.847  0.076 

47  46  F  115  10  T  0.933  0.892  0.040 

48  47  N  115  OR  0.922  0.894  0.028 

49  49  N  105  5  R  0.901  0.857  0.044 

50  50  N  105  5  W  0.880  0.857  0.023 

51  51  M  115  10  W  0.912  0.902  0.011 

52  52  M  105  5  R  0.917  0.885  0.032 

53  53  M  105  0  W  0.907  0.870  0.037 

54  54  F  105  OR  0.873  0.886-0.013 


0.918  30.800  0.800  0.962  2.893 
0.  957  31.000  0..  700  0.969  2.602 
0.969  30.500  0.800  0.953  2.764 
0.951  30.700  0.600  0.959  2.877 
0.974  30.400  1.100  0.950  2.953 
0.988  30.500  0.400  0.953  2.857 
0.965  30.900  0.800  0.966  2.838 
0.959  29.000  0.800  0.906  2.886 
1.015  29.900  0.800  0.934  2.696 
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APPENDIX  El 


T-TEST  RESULTS 


T-TE3T  FOR  WITHIN-RUN  FATIGUE/LEARNING  CURVE 


RN 

ACC-1 

VACC1 

ACC-2 

VACC2 

TIME-1 

VTIM1 

TIME-2 

VTIM2 

TACCY 

TTIME 

TCRIT 

CRIT? 

—  — 

1 

0.927 

0.036 

0.800 

0.016 

2.760 

0.  135 

2.780 

0. 144 

1.761 

0.118 

2. 101 

2 

0.964 

0.007 

0.955 

0.004 

2.560 

0. 147 

2.497 

0.110 

0.278 

0.388 

2.  101 

3 

0.955 

0.012 

0.918 

0.006 

2.629 

0.232 

2.454 

0.  151 

0.865 

0.894 

2.101 

4 

0.955 

0.005 

0.964 

0.004 

2.568 

0.286 

2.406 

0. 171 

0. 303 

0.760 

2.  101 

5 

0.945 

0.004 

0.973 

0.003 

2.745 

0.  166 

2.844 

0.117 

1.029 

0.587 

2.101 

6 

0.900 

0.007 

0.855 

0.005 

2.687 

0.  176 

2.718 

0. 194 

1.278 

0.  162 

2.  101 

7 

0.973 

0.003 

0.945 

0.012 

2.761 

0.  149 

2.817 

0. 181 

0.697 

0.309 

2.  101 

8 

0.936 

0.013 

0.982 

0.001 

2.767 

0.  138 

2.617 

0.147 

1.188 

0.887 

2.101 

9 

0.945 

0.012 

0.736 

0.024 

2.804 

0. 171 

2.726 

0. 122 

3.495 

0.455 

2.  101 

A 

10 

0.973 

0.002 

0.882 

0.007 

2.944 

0.207 

2.788 

0.111 

3.  131 

0.876 

2.  101 

A 

11 

0.982 

0.001 

0.964 

0.007 

2.742 

0.280 

2.618 

0.226 

0.632 

0.553 

2.  101 

12 

0.973 

0.007 

0.845 

0.007 

2.733 

0.  157 

2.846 

0.260 

3.478 

0.549 

2.  101 

A 

13 

0.973 

0.003 

0.927 

0.008 

2.841 

0.  194 

2.858 

0.167 

1.351 

0.085 

2.  101 

14 

0.982 

0.001 

0.964 

0.004 

2.843 

0.235 

2.823 

0.295 

0.816 

0.088 

2.  101 

15 

0.973 

0.003 

0.973 

0.  003 

2.795 

0.226 

2.532 

0.153 

0.000 

1.352 

2.  101 

16 

0.973 

0.003 

0.900 

0.011 

2.880 

0.  155 

3. 121 

0.270 

1.940 

1.165 

2.101 

17 

0.927 

0.010 

0.909 

0.  007 

2.750 

0.205 

2.767 

0.238 

0.452 

0.080 

2.  101 

18 

0.964 

0.002 

0.973 

0.003 

3.045 

0.337 

2.937 

0.414 

0.392 

0.393 

2.  101 

19 

0.973 

0.003 

0.982 

0.001 

3.092 

0.405 

2.992 

0.339 

0.419 

0.368 

2.  101 

20 

0.973 

0.002 

0.845 

0.008 

2.994 

0.241 

3.050 

0.258 

4.008 

0.253 

2.  101 

A 

21 

0.964 

0.004 

0.955 

0.002 

3.206 

0.394 

3.  107 

0.428 

0.381 

0.345 

2.  101 

22 

0.945 

0.004 

0.945 

0.004 

3.023 

0.302 

2.-941 

0.178 

0.000 

0.376 

2.  101 

23 

0.955 

0.004 

0.936 

0.005 

2.871 

0.271 

2.828 

0.  192 

0.614 

0.  199 

2.101 

24 

0.982 

0.001 

0.927 

0.005 

2 . 633 

0.  138 

2.876 

0.256 

2.236 

1.223 

2.  101 

A 

25 

0.991 

0.001 

0.982 

0.001 

2.967 

0.310 

3.007 

0.277 

0.632 

0.  165 

2.101 

26 

0.964 

0.004 

0.973 

0.003 

2.774 

0.210 

2.595 

0.  121 

0.343 

0.981 

2.  101 

27 

0.936 

0.008 

0.955 

0.007 

2.696 

0.274 

2.666 

0.208 

0.464 

0.  134 

2.  101 

28 

0.973 

0.002 

0.936 

0.003 

2.722 

0.  177 

2.768 

0.182 

1.606 

0.241 

2.101 

29 

0.982 

0.001 

0.945 

0.007 

3.004 

0.412 

2.896 

0.344 

1.265 

0.395 

2.101 

30 

0.991 

0.001 

0.991 

0.001 

2.900 

0.354 

2.931. 

0.340 

0.000 

0.119 

2.  101 

31 

0.973 

0.002 

0.873 

0.007 

2.751 

0.  184 

2.737 

0.254 

3.395 

0.070 

2.  101 

A 

32 

0.973 

0.007 

0.964 

0.007 

2.948 

0.307 

2.988 

0.228 

0.246 

0.  172 

2.  101 

33 

0.945 

0.007 

0.982 

0,003 

3,  101 

0.  184 

2.910 

0.162 

1.155 

1.024 

2.  101 

34 

0.991 

0.001 

0.955 

0.009 

2.740 

0.224 

2.667 

0.219 

1.185 

0.344 

2.101 

35 

0.955 

0.004 

0.891 

0.011 

2.993 

0.  145 

2.965 

0.211 

1.645 

0.148 

2.  101 

36 

0.982 

0.001 

0.973 

0.002' 

3.035 

0.397 

2.983 

0.308 

0.520 

0.  195 

2.101 

37 

0.964 

0.007 

0.864 

0.009 

2.754 

0.228 

2.600 

0.171 

2.530 

0.772 

2.  101 

A 

38 

0.945 

0.004 

0.964 

0.007 

2.799 

0. 139 

2.747 

0.  124 

0.559 

0.322 

2.  101 

39 

0.927 

0.005 

0.864 

0.020 

2.667 

0.  197 

2.655 

0. 160 

1.276 

0.067 

2.  101 

40 

0.918 

0,006 

0.927 

0.013 

2.803 

0.112 

2.759 

0.  102 

0.211 

0 . 305 

2.  101 

41 

0.945 

0.004 

0.864 

0.037 

2.629 

0.255 

2.545 

0.201 

1.287 

0.395 

2.  101 

42 

0.945 

0.007 

0.909 

0.012 

2.631 

0.  122 

2.718 

6.147 

0.845 

0.530 

2.  101 

43 

0.955 

0.009 

0,973 

0.003 

2.743 

0.084 

2.831 

0.101 

0.523 

0.647 

2.101 

t 


RN  ACC-1  VACC1  ACC-2  VACC2  TIME-1  VTIM1  TIME-2  VTIM2  TACCY  TTIME  TCRIT  CRIT? 
==  =====  =====  =====  =====  ======  =====  ======  =====  =====  =====  =====  #*-** 


44 

0.900 

0.014 

0.955 

0.004 

3.014 

0.  141 

2.959 

0.109 

1.297 

0.344 

2.  101 

45 

0.918 

0.016 

0.882 

0.020 

2.949 

0.  154 

2.815 

0.165 

0.610 

0.750 

2.  101 

46 

0,964 

0.007 

0.955 

0.004 

3.012 

0.138 

2.836 

0.130 

0.278 

1 . 073 

2.  101 

47 

0.964 

0.004 

0.955 

0.009 

2.594 

0.076 

2.611 

0.084 

0.259 

0.  134 

2.  101 

48 

0.982 

0.  003 

0.945 

0.009 

2.312 

0.  141 

2.747 

0.207 

1.069 

0.346 

2.  101 

49 

0.982 

0.001 

0.945 

0.007 

2.917 

0.221 

2.838 

0. 139 

1.265 

0.420 

2. 101 

50 

0.955 

0.007 

■  936 

0.010 

3.042 

0.  122 

2.898 

0.134 

0.441 

0.900 

2.  101 

51 

0.973 

0.002 

.  ‘>36 

0.013 

2.853 

0.056 

2.898 

0.  107 

0.938 

0. 355 

2.  101 

52 

0,955 

0.004 

0.  .  • 

0.004 

2.977 

0.223 

2.708 

0.214 

0 . 

1.287 

2.  101 

53 

0.973 

0.003 

0.9io 

0.008 

2.936 

0.  107 

2.816 

0.  132 

1.061 

0.776 

2.  101 

54 

0.945 

0.007 

0.964 

0.004 

2.763 

0.275 

2.565 

0.194 

0.559 

0.911 

2.  101 

KEY! 

RN;  Run  number 

ACC- 1 :  Group  mean  accuracy  Tor  the  first  one-third  responses 
V ACC  1 :  Variance  for  ACC-1 

ACC-2:  Group  mean  accuracy  for  the  last  one-third  responses 
VACC2:  Variance  for  the  ACC-2 

TIME-1;  Group  mean  time  for  the  first  one-third  correct  responses 
VTIill;  Variance  for  TIME-1 

TIME-2:  Group  mean  time  for  the  last  one-third  correct  responses 

VTIM2:  Variance  for  TIME-2 

TACCY:  Student’s  t-value  for  accuracy  t-test 

TTIME:  Student's  t-value  for  response  time  t-test 

TCRIT:  Critical  value  for  "t"  using  18  d.f. 

CRIT?:  "A"  indicates  accuracy  was  statistically  significant 

"T"  indicates  response  time  was  statistically  significant 


APPEND X  X  ES 
MANOVA  RESULTS 
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MULTIPLE  ANALYSIS  OF  VARIANCE 

— =H0TELLIN6'S  T=====  =======  UNIVARIATE  ANOVA 


TREATMENT 


T-VALUE 

F-VALUE 

SIGNIF 

VOICE  TYPE  IVTYFEI 

.07322 

8.73148 

6.4E-7 

BACKGROUND  CBK6RD1 

.02784 

6.65354 

0.001 

SI6/N0ISE  CSNRATT 

.06423 

7.66580 

4.4E-6 

PRECURSOR  CPREC] 

.21211 

25.29370 

0.000 

SUBJECT  NO. 

4.3148 

114.34263 

0.000 

VTYPE  X  BKGRD 

.02308 

2.75176 

0.027 

VTYPE  X  3NRAT 

.02850 

1.69954 

0.094 

VTYPE  X  PREC 

.04802 

2.86307 

0.004 

BKGRD  X  SNRAT 

.01753 

2.09065 

0.080 

BKGRD  X  PREC 

.03193 

3.80768 

0.004 

SNRAT  X  PREC 

.03951 

2.35561 

0.016 

VTYPE  X  BK6RD  X  SNRAT 

.01065 

0.63495 

0.749 

VTYPE  X  BKGRD  X  PREC 

.03351 

1.99817 

0.044 

VTYPE  X  SNRAT  X  PREC 

.09713 

2.89571 

0.0001 

VTYPE  X  BKGRD  X 

SNRAT  X  PREC 

.07106 

2.11843 

0.006 

—ACCURACY— 

-RESPONSE  TIME- 

--SIGNIF 

?— 

F-VALUE 

SIGNIF 

F-VALUE 

SIGNIF 

HOT  i 

ACCY 

TIME 

6.1737 

0.002 

11.2024 

1.8E-5 

YES 

YES 

YES 

13.1604 

0.0003 

0.0652 

0.799 

YES 

YES 

No 

14.5739 

7.2E-7 

0.5678 

0.567 

YES 

YES 

No 

24.6654 

0.000 

26.1132 

0.000 

YES 

YES 

YES 

37.3532 

0,000 

189.5643 

0.000 

YES 

YES 

YES 

5.1847 

0.006 

0.3994 

0.671 

YES 

YES 

No 

0.6057 

0.659 

2.7268 

0.029 

No 

No 

(YES) 

1.5073 

0.199 

4.4369 

0.002 

YES 

No 

YES 

0.8322 

0.436 

3.4400 

0.033 

No 

No 

(YES) 

3.9409 

0.020 

3.6643 

0.026 

YES 

YES 

YES 

1.4881 

0.205 

3.0836 

0.016 

YES 

No 

YES 

0.3622 

0.836 

0.9000 

0.464 

No 

No 

No 

3.1392 

0.014 

0.8467 

0.496 

YES 

YES 

No 

1.9641 

0.049 

3.9315 

1.6E-4 

YES 

YES 

YES 

2.9185 

0.003 

1,4467 

0.175 

YES 

YES 

No 
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UNIVARIATE  ANALYSIS  OF  VARIANCE 
Using  Unadjusted  Response  Times 

UNIVARIATE  SIGNIFICANCE  FDR  DISCRIMINATION 


- ACCURACY - 

F  Si gn if 


TREATMENT 

Val  ue 

of  F 

VOICE  TYPE 

8,506 

0.000 

BACKGROUND 

35.322 

0.000 

S/N  RATIO 

26.533 

0.000 

PRECURSOR 

38.674 

0,000 

SUBJECT  SEX 

101.573 

0.000 

--RESPONSE  TIME-- 


F 

Signif 

Value 

of  F 

Signif  ? 

43.767 

0.000 

ACCY 

TIME 

0. 131 

0.717 

ACCY 

2.205 

0.111 

ACCY 

152.937 

0.000 

ACCY 

TIME 

82.172 

0.000 

ACCY 

TIME 

-GROUPING  BY  SHEFFE  PROCEDURE 
(Unadjusted  Response  Times) 


- . ACCURACY 

Subset  1 
MALE 

Subset  2 
FEMALE 
MACHINE 


-  VOICE  TYPE 

Mean 

.9460 

.9318 
.9281 


. RESPONSE 

Subset  1 
FEMALE 
Subset  2 
MALE 

Subset  3 
MACHINE 


TIME . 

Mean 

2.7486 

2.8182 

2.8612 


.  ACCURACY 

Subset  1 
105  db 
SUBSET  2 
115  db 


-  BACKGROUND  . 

.  . RESPONSE  TIME 

Mean 

Subset  1 
.9464  115  db 

105  db 


.9242 


Mean 

2.8075 

2.8111 


.  ACCURACY 

Subset  1 
10  db 
Subset  2 
5  db 

Subset  3 
0  db 


—  S/N  RATIO 
Mean 
.9517 
.9358 
.9184 


. RESPONSE 

Subset  1 
10  db 
0  db 
5  db 


TIME . 

Mean 

2.7969 
2.8091 
’  2.8222 


.  PRECURSOR  . 

- ACCURACY -  *  - RESPONSE  TIME 

Mean  Mean 

Subset  1  Subset  l 

REPEATED  .9582  TONE  2.6882 

Subset  2  Subset  2 

TOwe  .9274  REPEATED  2.8549 

VOICE  .9203  VOICE  2.8839 


<=»!=■  F’EI'-JD  X  X 


ANOVA  RESULTS 
BY  SUBJECT  SEX 


UNIVARIATE  ANALYSIS  OF  VARIANCE 
BY  SUBJECT  SEX 
(Unadjusted  Response  Times) 


UNIVARIATE  SIGNIFICANCE  FOR  DISCRIMINATION 
- ACCURACY -  —RESPONSE  TIME— 


F  Signif  F  Signif 


TREATMENT 

Value 

of  F 

Value 

of  F 

Signif  ? 

SUBJECT  SEX 

101.573 

0.000 

82. 172 

0.000 

ACCY 

TIME 

MEN:  VOICE  TYPE 

1.641 

0.  194 

7.273 

0.001 

TIME 

WOMEN:  VOICE  TYPE 

6.987 

0.001 

45.388 

0.000 

ACCY 

TIME 

GROUPING  BY  SHEFFE  PROCEDURE 
(Unadjusted  Response  Times) 


.  SUBJECT  SEX  . 

. -  ACCURACY . —  . RESPONSE  TIME . 

Mean  Mean 

Subset  1  Subset  1 

MEN  .9583  <  WOMEN  2.7719 

Subset  2  '  Subset  2 

WOMEN  .9199  MEN  '  2.8631 

.  MALE  SUBJECTS  . 

. ACCURACY .  . RESPONSE  TIME . 

Mean  Mean 

Subset  1  Subset  1 

MALE  VOICE  .9644  FEMALE  VOICE  2.8269 

FEMALE  VOICE  .9562  MALE  VOICE  2.8579 

MACHINE  VOICE  .9544  Subset  2 

MALE  VOICE  2.8579 

MACHINE  VOICE  2.9047 

.  FEMALE  SUBJECTS  . 

. ACCURACY  — — .  . RESPONSE  TIME . 

Mean  Mean 

Subset  1  Subset  1 

MALE  VOICE  .9337  FEMALE  VOICE  2.6941 

Subset  2  Subset  2 

FEMALE  VOICE  .9155  MALE  VOICE  2.7908 

MACHINE  VOICE  .9106  Subset  3 

MACHINE  VOICE  2.8309 
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append  i  x  e:s 
ANQVA  RESULTS: 
ADJUSTED  RESPONSE  TIMES 
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UNIVARIATE  ANALYSIS  OF  VARIANCE 
**  Response  Times  Adjusted  for  Warning  Length  ** 


UNIVARIATE  SIGNIFICANCE  FOR  DISCRIMINATION 


- ACCURACY - 

F  Si g  n  i  f 


TREATMENT 

Value 

of  F 

VOICE  TYPE 

8.50£> 

0.000 

BACKGROUND 

35.322 

0.000 

S/N  RATIO 

26.533 

0.000 

PRECURSOR 

38.674 

0.000 

SUBJECT  SEX 

101.573 

0.000 

MEN:  VOICE  TYPE 

1.641 

0.  194 

WOMEN:  VOICE  TYPE 

6.987 

0.001 

— RESPONSE  TIME— 


F 

Signif 

Value 

II 

Ul  11 

II 

'4-  1! 

O  || 

If 

Signif  ? 

18.511 

0.000 

ACCY 

TIME 

0.184 

0.668 

ACCY 

1.609 

0.201 

ACCY 

1763.836 

0.000 

ACCY 

TIME 

73.865 

0.000 

ACCY 

TIME 

13.657 

0.000 

TIME 

7.064 

0.001 

ACCY 

TIME 

GROUPING  BY  SHEFFE  PROCEDURE 
(Adjusted  Response  Tiroes) 


VOICE  TYPE 


— .  ACCURACY  - 

Mean 

. RESPONSE  TIME  - 

Mean 

Subset  1 

Subset  1 

MALE 

,  9460 

MALE 

1.0350 

Subset  2 

Subset  2 

FEMALE 

.9318 

MACHINE 

1.0952 

MACHINE 

.9281 

FEMALE 

1.1110 

.  BACKGROUND  - 

ACCURACY  - .  . RESPONSE  TIME 


Subset  1 

Mean 

Subset  1 

Mean 

105  db 

.9464 

115  db 

1.0778 

SUBSET  2 

105  db 

1.0824 

115  db 

,9242 

ACCURACY 


S/N  RATIO 


RESPONSE  TIME 


Subset 

1 

Mean 

10  db 

.9517 

Subset 

2 

5  db 

.9358 

Subset 

3 

0  db 

.9184 

Subset  1 

Mean 

10  db 

1.0695 

0  db 

1.0781 

5  db 

1.0929 

.  PRECURSOR  . 

ACCURACY .  . RESPONSE  TIME 


Subset  1 

Mean 

Subset  1 

Mean 

REPEATED 

Subset  2 

.9582 

REPEATED 

Subset  2 

.  6770 

TONE 

.9274 

VOICE 

1.2648 

VOICE 

.9203 

Subset  3 

TONE 

1.3133 
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ACCURACY 


SUBJECT  SEX 


Mean 

Subset  1 

MEN  .9583 

Subset  2 

WOMEN  -9199 


- RESPONSE  TIME  - 

Mean 

Subset  1 

WOMEN  1.0415 

Subset  2 

MEN  1.1358 


.  MALE  SUBJECTS  - 

- ACCURACY -  - RESPONSE  TIME 

Mean  Mean 

Subset  1  Subset  1  , 

MALE  VOICE  .9644  MALE  VOICE  1.0763 

FEMALE  VOICE  .9562  Subset  2 

MACHINE  VOICE  .9544  MACHINE  VOICE  1.1416 

FEMALE  VOICE  1.1899 

.  FEMALE  SUBJECTS  - 

- ACCURACY -  - RESPONSE  TIME 

Mean  tlean 

Subset  1  S^set  1  , 

MALE  VOICE  '  .9337  MALE  VOICE  1.0065 

Subset  2  Subset  2 

FEMALE  VOICE  .9155  FEMALE  VOICE  1.0560 

MACHINE  VOICE  .9106  MACHINE  VOICE  1.0628 


^f-f=-e:!ni3D  x  x  r=- 

SUBJECT  QUESTIONNAIRE 
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Vo±  ce  —  War"ni  ng 
QUEST X ONWA I 


Thank  you  -For  participating  in  our  experiment  to  help 
determine  response  accuracy  and  speed  to  different  voice 
■  types.  As  we  explained  at  the  beginning,  results  -from  this 
experiment  will  be  use-ful  for  designing  better  alerting 
systems  for  our  crewmembers,  thereby  improving  aircraft 
safety  and  survivability. 

No  experiment  dealing  with  human  reaction  is  really 
complete  unless  the  personal  reaction  of  the  subjects  is 
also  taken  into  account.  To  help  us  in  our  evaluation,  we 
would  appreciate  it  if  you  would  please  complete  the 
following  questionnaire  as  completely  as  possible. 

Please  circle  the  answer  which  most  appropri atel y 
reflects  your  feelings  and  reactions.  If  none  of  the 
answers  seems  appropriate  for  you,  please  mark  "other"  and 
explain. 


Some  of  the  answers  will  request  a  "scale"  of  your  reaction,  rather  than 
a  direct  answer.  In  marking  your  answers  for  those  type  questions,  please  use 
the  following  scale: 


(a) 

Not  at  all, 

completely  disagree, 

extremely 

negat i ve , 

etc . 

(b> 

A  little, 

mildly  disagree, 

slightly  n 

eqative, 

etc . 

(c) 

Neutral , 

middle-of-the-road, 

ambivalent 

i 

etc . 

(d) 

Somewhat , 

mildly  agree, 

mildly  pos 

.  i t i ve , 

etc . 

(e) 

Very  much, 

strongly  agr-ie, 

extremely 

posi t i ve , 

etc . 

(f ) 

Other  (please 

explain) 
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NAME  (Optional) _ 

Seat/Module  Number _ 

1.  The  primary  (tracking)  task  kept  me  occupied  throughout  each  session. 

(a)  (b)  1  (c)  (d)  (e)  9  (f) 

2.  There  was  adequate  time  to  respond  to  the  primary  task. 

(a)  (b)  1  (c)  2  (d)  6>  ;e>  1  (f) 

3.  I  knew  exactly  when  a  particular  session  should  end,  either  through  timing 
or  by  counting  the  number  of  warnings. 

(a)  2  (b)  2  (c)  (d)  6>  (e)  (-f ) 

4.  I  could  anticipate  which  "warning"  button  would  be  called  for  next. 

(a)  6>  (b)  1  (c)  (d)  2  (e)  1  (f) 

5.  I  could  anticipate  when  the  next  "warning"  would  be  called  for. 

(a)  4  (b)  4  (c)  (d)  2  (e)  (f) 

b,  When  required  to  respond  to  the  "warning",  I  completely  ignored  the  primary 
task  (tracking)  until  after  I  had  responded  to  the  warning. 

(a)  2  (b)  1  (c)  1  <d)  6  (e)  (f) 

7.  To  which  voice  type  did  you  feel  most  comfortable  responding? 

(a)  Male  a  (b)  Female  4  (c>  Machine 

8.  To  which  voice  do  you  think  you  responded  most  accurately? 

(a)  Male  6>  (b)  Female  4  (c)  Machine 

9.  To  which  voice  do  you  think  you  responded  most  quickly? 

(a)  Male  4  (b)  Female  <b  (c)  Machine 

10.  Which  voice  did  you  prefer? 

(a)  Male  4  (b)  Female  <6  (c)  Machine 
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